Your First Steps Into Personalization

more people to come in because this is going to be an interactive session, and this is not going to be a monotonous webinar. Johann has put a lot of effort into building this workshop, so it’s going to be really fun. I see someone has raised hands, Laura, I see that you’ve raised your hand. Let me know if you have a question. You can type in the questions box if you have a question.

Cool then. I see a good number of people have come in now, and, we’ll officially start the session. So, yeah, welcome once again to what’s going to be another insightful session at the VWO webinars. My name is Vipul, and I am the Senior Marketing Manager at VWO. And, I welcome Johann, from AWA Digital, who will be basically moderating this entire session and sharing insights that you can implement directly at your respective businesses plus hands-on, tactics that’ll help you learn more about how to get started with personalization.

So Johann is there. You can see him. Before Johann starts with the presentation, I just wanted to again announce that this is going to be an interactive workshop. Right? So in the middle of the session, you know, after Johann has shared, a few of the insights, he will share, he will display a URL for the Miro board and, we’ll give you a minute to join the Miro board, and then you’ll be able to, interact and and become a participant of this workshop. So we’ll let you know when the time comes. So yeah, that is all. If you have any questions during the course of the session, just put them in the questions panel. And I’ll be happy to unmute you so that you can ask your questions directly to Johann with more context.

So also we can make it a more engaging and interactive, session that way. With that, I’ll jump off the stage, and hand over the mic to Johann.

Johann Van Tonder:

Thank you, Vipul, and welcome, everyone. I hope you can see my screen now. This as we said, it’s not a slide deck. This is a mirror board. I am going to be scrolling through it.

And at some point, pretty soon, things are gonna get relatively interactive. I hope you’ll join me. You’ll play along when I share the link with you. I have been doing experimentation and optimization now for close to 15 years. I spent a long time in corporate, then ran an eCommerce store for a while.

For the last 10 years, I’ve been an agency working at AWA Digital, which is a specialist CRO agency, an eCommerce CRO agency. So I’ve been in the woods for a long time. And my role in the agency gives me, a high-level vantage point into experimentation and CRO programs around the world, retailers, and eCommerce businesses around the world. And I speak to teams, both my own clients, but also non-clients, frequently.

And one of the themes that keep coming up consistently and it’s getting increasingly so is personalization. There’s a lot of noise around personalization. There’s even more confusion. And, hopefully, after the next 30 minutes or less now, what I want to ask quickly is, just tell me where you are in the world. Just a couple let’s see if we can get their interaction going and make sure we can find the right box.

They should be in your control panel, the questions box. And let’s see if you can get that to work. So just type the city where you are, so we can see if we get that working.

One or two I’d like to see, and then, can people find the question box? In the control panel, Vipul, do people find the question box?

V:

It’s the daily control panel, guys. And, if you’re not able to view it, just, you know, you can raise your hand. and, we’ll try to see if the issue can be resolved for you. On the right, the questions panel, you’ll see that there’s a box, by the name of by the title of questions and, It would be interesting to see, where you guys are joining in from. I think it’s disabled for me. Maybe you can try this? Can somebody try and unmute themselves and, just shout out? See if we get this. Just

raise your hand, and I’ll unmute you. Yeah. So, Cheng is from Malaysia. Hi, Shane.

JVT:

Oh, okay.

V:

Great to navigate. Okay. Alright. Thanks. The questions box is working. Just that, we have a bit of a shy audience today, and, they’ll definitely, yes. The spots are coming in now.

JVT:

Right. So, this is not the time to be shy. It’s completely anonymous. And, you know, the point here is to, get you, to actually do some of the exercises. It’s gonna be very lightweight, but it is gonna be interactive.

And the reason for that is this kind of stuff you learn by doing, so there’s an enormous appetite for personalization, but the struggle is as big as an appetite is. The struggle is even bigger. In fact, Gartner, you may have seen the stat Gartner claims that 80% of marketers who are doing personalization will, abandon personalization efforts in the next 3 years by 2025. Now that’s massive, and I’m not surprised to hear that.

Because I see how people, are struggling with it. And this is a quote from Peep Laja, and I completely agree. Basically, what Peep is saying is that there’s this overreliance on tools. You know, people think that tools will be the answer. And I can understand how people get to that point.

Certainly from the teams that I’m speaking to, and maybe this is your experience as well, to get started is incredibly hard. I mean, to execute is hard, but where do you start? How do you make that start in tools? The promise of hyper-personalization tools is that you know, with a quick integration, you can be out of the blocks and you can start doing hyper-personalization. So that appeal is quite high, but you quickly get into a situation where the tools drive the strategy.

And as Peep says, you know, it’s still that human behind the tool that needs to make these decisions. Now I’m gonna show you some examples of personalization before we start getting into it. And the first one here, I think, is quite predictable, and it’s quite lame. Perhaps it’s Amazon. For me, personally, as a consumer, I actually think that Amazon is one of the best examples.

And the reason for that, as I said, from a consumer perspective, is how it allows me to discover things that I would not have known. So in this case, books, the algorithm knows what I like. In this case, there’s a lot of statistics and user research and you know, that side of the science and because that’s what I read. And, the algorithm knows that. And so I constantly am buying books and, you know, products that I would otherwise not have known about. So it’s very useful, very convenient.

This one is from Tommy Hilfiger. This is quite a common theme in eCommerce, where if you arrive on the site the first time, you see that screen on the left, that’s the home screen. And then once I’ve interacted with, in this case, the men’s category, the next time I come back to the site it recognizes me. It says, welcome back. And it takes me straight to, well, this is the home page, but it gives me a shortcut, a short path to that main strategy because that’s what, the interest that I’ve shown, and it’s picking up on that.

This is still in the same theme. There are a couple of these. And usually, a lot of personalization. In fact, I’d say that almost all personalization targets returning visitors for obvious reasons. But we’ll get into the granular detail a little bit more, and break it down a bit more later.

Here’s another simple one. The first time I arrive at the site is on the left. Once I’ve registered and they know a bit about me, the next time I come back is a more personalized message, and the more they hear about me, the more they get to know about me, and the more they can target that journey. Now we take that a step further, with another eCommerce site, and this experience allows me to pick up where I left off the last time. So the last time I was on the site, I looked at those products.

So the next time I come back, they bring me straight into that journey because I browsed these products, and visited these products in a previous session. It tries to improve the experience for me by giving me a shortcut to that. So that’s kind of the theme that you see coming up a lot here. I’ve got 2 more examples, and then we’ll start looking at how to put this all together. This one I think is from Farfetch. And, when I’m on this page and I click on the button, what’s my size, which, of course, is quite a ubiquitous pattern on this product page, product pages on fashion sites, and clothing sites.

It gives me this box, and it takes you through a sequence of steps where you give them your preferences, your you know, your length, your weight, your style, and a range of questions, that not only are used in this particular session. It helps me to find the particular size that I need for the product that I’m buying, but that data is then subsequently used to personalize my journey. So it’s quite a neat way of weaving that in because it’s in context that this data has gathered. So I don’t mind as a user giving them this data and they’re not interrupting my journey. It is, for my benefit.

And, you know, it’s entirely acceptable to give him his data. So it’s a very clever way of doing it. Stitch Fix- you may have heard of this, in fact, the entire business model. If you haven’t been on Stitch Fix, go after the call and go and play around on the site and get a sense of what they’re doing, and the entire business model is based on this. So when you can’t do it any other way, when you click on that button, take your style quiz. It takes you through this series of questions where it gathers all this data on you, and then it will tailor, the offers that you’ll see now, and this gets fairly complex.

I want to dumb it down today. The entire session today, the next 30 minutes or so, I wanna show you just how easy it is to get started before we get into this kind of complex, relatively complex scenario that I’ve just painted out there. Now I’m assuming 2 things. I’m assuming that you know A/B testing. You’re familiar with experimentation and A/B testing.

And I’m also assuming you are at your journey’s start. You know, this is gonna be a very basic session. It is the first step into personalization. If you’re looking for tips and tricks and advice on how to go from advanced personalization to more advanced personalization, this is not the session for that today. We’ll have another chat about that. 

But with A/B testing, you know the advantages and disadvantages. And there’s a table that I love that really explains the difference between the 2 A/B testing and personalization. So I’ll run through it quickly. I don’t want to spend too much time here. I want to start getting interactive.

But with A/B testing, you’ve got, in this scenario, 3 different variations. And what we’re doing and A/B testing remains the best way to optimize for the entire audience. And you can see that here. So that average is at the very bottom of this table. In this case, we would have chosen variation too. For the entire audience, that’s the best variation.

But when you start drilling into the underlying segments, then you can see that for group A, variation 1 would actually have been the best choice, and for group C, variation 3 would have been the best choice. And would they be testing if we tested the entire audience, we would have settled on variation 2. And, you know, it’s, as I say, it’s a good starting point, but at some time, at some point, do you want to start, maximizing that return? You would have to start drilling deeper, and that’s the right point to move over into personalization and to do the 2 in parallel. It doesn’t mean you stop doing A/B testing, but it means you start adding, personalization onto it and you start, segmenting it deeper. Now this, is what we’re looking at here at the moment, if you wanted to answer the question, how do I get started?

This is actually a very good way to get started to identify those audiences. Personalization is all about audiences. We’ll talk a lot about that today. This is one method to help you identify who those audiences might be, but it gets even easier than that. I just wanna say a word or 2 about the definition of personalization because as I said at the beginning, there’s a lot of confusion about it.

There’s a lot of noise around this. And on LinkedIn recently, David Mannheim is writing a book on this. We’re all looking forward to that. It’s gonna be great. and he’s been, discussing this a lot on LinkedIn, and there’s been a lot of debate, even in the industry, among product leaders, among people who are at the top of this industry about exactly what is personalization.

So if we as an industry struggle to, you know, align on that definition, you know, that shows you just how confusing it could be. The one important point that I wanna bring home today is that when we talk about personalization, the urge is, the temptation is, to limit that to 1-to-1 personalization, a hyper-personalized experience where each user is getting a unique experience. Now that is just one form of personalization. On the other side of the scale is one to all. Right?

That’s A/B testing. That’s where you test the entire audience as we said, but there’s this entire spec in the middle where you’re able to play. And some would argue this bit in the middle here is actually segmentation. It’s not personalization. My response to that is, it’s semantics.

It doesn’t matter. For our purposes and in the context of what we’re talking about today, helping you to move forward in personalization. It not only doesn’t make sense to play across this entire spectrum. But the closer you move to the far right of the spectrum, the further you move towards 1-to-1, the more difficult it becomes, the more challenging it becomes, the more expensive it becomes, and the less likely or the more difficult it is to achieve ROI. The more likely it is to burn out, for this to fizzle out.

In fact, when Gartner mentioned that stat, that 80% of marketers want to abandon personalization, this is one of the core reasons that their nature is the lack of ROI. So really where you wanna start today, if you haven’t yet, is here, with one-to-many. I have spoken to a lot of teams, and they’ve made it really difficult for themselves by starting here or even here. And then things fizzled out. And what I’ve advised is when they’ve started there is to take a few steps back to zoom out and to really just start playing in this area here.

And, today, I wanna show you, you know, how you can do that. How you get started. Of course, personalization relies on data. So it starts with data. You need a data point to bind these users together.

This is maybe another distinction I should make if you run an A/B test on a page template, the home page, on a PDP, or check out or whatever, that’s not personalization. It starts becoming in my view personalization as soon as it’s user-based. And there’s a variable, around which you bind those users together. And that’s the data point. And, I’ll show you a few of those examples.

And this is where I’d like you to join me on this mirror board, and we’ll start, making this a little bit more interactive. So I don’t know if you can share this URL as well, but, you can type that into your browser, and I’ll give you a minute or 2 just to join. I can see when you join, I really encourage you to, just follow that URL, and I’ll give you a minute to join there. Vipul, can you add it to the chat as well?

V:

Yes. I’ve dropped the URL on the chat as well. So, I think everyone can view the URL. So you just need to, I think, someone who has joined in, and this is an interesting workshop.

JVT:

I see something. And, it’s completely anonymous. As you can see there, it’s a visiting visionary is 1. You won’t be identified. Anything you write here, nobody will be able to connect it to you or your business.

If you’re serious about this, if you wanna make a head start, I really encourage you to play along with me. Now let me give it another few seconds. I see there are a couple of people, maybe 1 or 2 more want to join, and we’ll be another 20 minutes or so but, from this point, quite practical. Right?

So I’ll continue talking, but you’re welcome to join us. So here on what you can see in front of you, and I’ll bring everyone to me. So if you’re on the board, you should now see what I’m seeing. And if you’re watching my screen, well, that’s easy. Now this is, by no means an exhaustive list, but it’s an example of the kind of data points that we use to bind users together to execute personalization.

And you can see on the far left, it’s quite low sophistication. We’re talking about device, browser, location and you have all seen that. You know, if you visit a site from, say, the UK, versus the United States. You might get a different experience. You might get a localized version of the site. And in the middle, it starts getting a little bit more sophisticated. By no means sophisticated yet, these are the kind of data points that all of us have available without complicated tools without expensive tools. The examples that we showed earlier, the examples that I took you through, a lot of that had to do with pages. I visited products that I browsed.

And then on the far right, it starts getting more granular third-party data, 1st party data, and then zero-party data. So this is quite interesting zero party data. The difference between 1st party and zero party data is, that 1st party data comes directly from the user, but they don’t know it.

Zero-party data is like that example I showed from Farfetch where the user actually responds to a quiz and or something similar. It could be a survey or anything where the user knows they’re being asked for data, and that data is then used to personalize the experience. and you’ve got, access to this without any expensive tools. Now how do we get started? How do we move from that to, getting our first personalization live?

This is where I hope you’ll play along with me. I’m gonna ask you to create your own personalization, but before you do, let me explain how you do that. We’re gonna really dumb it down. And the reason for that, one of the main issues I see with personalization is people overcomplicate it. And when I speak to David, who’s writing the book, I get the same sense from him.

There’s this is part of the narrative around personalization. We assume because it’s personalization, it has to be complex. I’m gonna show you just how easy it is to get started. And if you’ve taken a jump or 2 ahead of yourselves, this is a good way to get back to it. So okay.

Let’s, think of some examples. I’m gonna add a sticky note here, and this is what I’d like you to do shortly. So I’m gonna think of, some audiences here. I want to come up with 2 audiences. So this is a subset of my entire audience, and remember it’s user-based.

So let’s say, I’m a clothing retailer, and we’ve seen some of those examples. So I’ve got people to buy men’s clothing. And I’ve got people who buy women’s clothing, and then maybe I’ve got kids as well. So those are the 3 categories on my side. So it makes sense from a business perspective, from a strategic perspective, even without any data, just knowing my business.

My gut, my intuition, my guess would be that these are the different segments that are important at a very high level. And if I’ve gone more granular, then I’d want to zoom back and start here. Now, also, at this point, want to venture a guess as to how big are these audiences. So men, let’s say it’s medium size. Women are the largest. And this, I’m thinking of an actual client

where there are maps on to that client. So now I’ve got a rough indication, and this is just a hypothesis at this stage. As A/B testers, experiments, we all know what that is. This is my best guess, my indicated guess. I might be wrong, but I’ll quantify this later.

One more example. Let’s say I am a retailer, and this is another one of my clients, I’m looking specifically at app users, and I think that a core group of users are weekly shoppers. So they buy the same groceries every week. And then another one is, let’s say, vegan. And I think vegan is quite a small audience relative to, you know, those weekly shoppers.

The weekly shoppers, it’s your trolley in the supermarket. That’s what I buy in the app. And maybe I’ll do one more just to give you a sense of this is also from a real client. They sell baby goods like, strollers and primes and bushes and clothing and that sort of thing. There are 3 personas, the first one is an expecting mother, and the second one is young parents. And the third one is what we call gifted. Now this could be grandparents or uncles and aunts. And then if I were to put sizes to that, this would be medium. This would be large, and this would be the smallest.

When I say size, just to be clear, this is the size of the audience. In other words, how many users, the portion of the audience, of the entire audience, how it breaks down. If you’ve done this, you’ve made a really good start. You are ahead of many people who are struggling with personas, sorry, with personalization, and this is really the first exercise you wanna do without any data and you probably have the data in the back of your mind. It’ll be data-informed.

But just think from your business perspective. What are those core audiences? And maybe you can think of just one for now. That’s fine. If you don’t want to use your own business, this is entirely anonymous, but if you don’t wanna use your own business, think of another business, think of any site that you visit. And I’m gonna give you 3 minutes.

Put the timer on. Maybe we’ll make it 4 minutes. I’ll play a bit of a jam. And then, you can grab a sticky note. You can grab a sticky note at the top there, or you can just right-click like this. And, there should be add sticky note. And then just, scroll down. You can add your sticky notes here or do it at home on a piece of paper, but this is what I’d like you to do. And, we’ll give it 4 minutes.

If you have any questions, you can shout out. And then we’ll soon take this to the next level. So this is one step out of three steps. Don’t worry about breaking the board. You can’t break the board.

So you’re welcome to add your sticky notes, at the bottom there, or, as I say, on a piece of paper. As long as you do this exercise, think about your audiences in your business.

V:

2 minutes left. Looks like the existing architecture is up to something. That’s great to see.

JVT:

Hey. Well, I think I’ll stop it there. Okay. Right. So we’ve got, some sticky notes here.

And, I hope the rest of you have done it, scribbled it on a piece of paper. And, if you want to I’m not gonna pick on anyone, but if you want to just jump in and call out your audience, you’re welcome to do that. If you want some feedback you can save it right into the end, but there’s absolutely no pressure. This is for you. This is your exercise.

The point is this is how you get started. So far, what we’ve done is we’ve identified an audience, and we’ve done one of two ways. We haven’t used an expensive tool. We’ve thought about our business context, and I’ve given you 3 examples there. And it’s really just a strategic perspective.

We all know our businesses. We all know at a high level what those core groupings might be. I want to encourage you and challenge you not to make it as simple as returning visitors. Of course, that is a segment. And, you know, that’s a segment that you might get a lot of mileage out of.

But when you do this exercise, try and go a little bit beyond that, you know, try and think strategically about what those audience examples could be, like the ones I’ve mentioned there. So that’s one way of doing it. The other way of doing it is when you run an A/B test, you dig into those segments and you can try and use that for inspiration. Okay. Now I’ll go over this quickly because I think this is fairly obvious.

You’ve made an assumption about how big that audience is. You wanna quantify that and that you can do in your analytics, whether it’s GA or, Adobe or whatever you’re using. And, you know, you build a segment around what you’ve identified, and you just quantify that audience and get a relative size, and that’s really very important. I think it’s intuitive to understand this if it’s especially if you understand A/B testing, the bigger your sample, the bigger the size of the audience, you know, the more you have to work with. And this is what brings us to the prioritization model. 

Rice is quite a common model, and you may have come across this. It’s very popular, but I do wanna go through it quickly because there are some nuances, some tweaks, in the way we apply it here. So that is the number of people, how many users in that audience and that’s where the small, medium, large, and the quantification comes in. And as I’ve explained, that’s quite important, and that’s really a starting point.

All things being equal, you would start with an audience that gives you the bigger reach. The impact can be interpreted in one or two ways. It could be simply how much impact you expect to get from this experience. So how much whatever you’re measuring, whether it’s revenue or conversion rate or, user experience, or however you find that, and however you measure it, it could be that. That’s a bit of a guess. And, if you’ve done enough A/B tests and I’ve done over 3000 in the last 10 years, so, what I do know is we’re wrong more often than we write.

So I didn’t like doing it that way. What I’d like to do with impact is just like we’ve done with reach, we’ve quantified it. We’ve looked at the percentage of users that fit into that bucket. I like to look at the contribution to revenue of that segment in an eCommerce context, and I like to use that as an impact.

Confidence is a little bit different than the normal application in the Rice model. Confidence, in this case, for me, is about data. How confident we are about the availability of that data? Can we actually access it? And do we trust it?

And then if it, I think, is, what we all understand it to be is how long it will take, how much dev time, and how much cost. And then we, let me find my pen. What we would do is it’s reach times impact times confidence divided by effort, and that gives us a score. And that helps us to prioritize these segments. But all things being equal, it’s gonna be the higher reach and the higher impact.

That’s really the way to look at this. Now there’s one more thing that I need to point out. One more practical exercise, but before I get there, I need to take a quick detour, and I’ll keep it quick because I assume that everyone knows this. Just to remind us of what a hypothesis, a good hypothesis looks like.

And this is a framework from Craig Sullivan. He’s published, this on Medium. So you can go and Google it afterward and look at this. If you’re not familiar with it, why I like this is Craig worked for long. A lot of effort is going into this.

You work with many other people, and that’s gone through various iterations. Now, basically, a hypothesis, as you know, is that x will lead to y and we’ve got the data there based on our observations. We believe that x, the change, for the population, and this is quite important in personalization is to have that within and then will cause the impact. So we’ve just spoken about that.

This is reach and impact. and then the rest, I’m gonna make the assumption. You know, what I wanna show you is how to bring it home and why this is so important. With the reach model, you can see how these 2 overlap with each other. Right?

So reach, how many people will be affected is that population bit. Right? So it’s this bit over here, this top bit. Impact is the second bit. Right?

So it maps directly onto your hypothesis and the rest we’ve spoken about. And now we get to the last bit. And, again, I invite you to play along, but this is the exercise that really brings it all together, how you move from that audience that you’ve identified to the point where you’ve got something to show them. You’ve got an idea. So I’m gonna show you an example quickly. And, again, this is from a real case study that I’m working on at the moment, one of our clients, and I’m gonna go back to. So we’ve already identified our audiences, right, in the earlier exercise.

I asked you to identify those high-level audiences, and one of them I had was the app users, who are the weekly grocery shoppers. Right? So that’s one of our audiences. And we’ll have, you know, 3 or more. How many other audiences we’ve identified? Now this is a very important part of the exercise. Don’t skip this. This is the next step. And where I see a lot of things go wrong is skipping this step because what I often see happening is that teams get told to do personalization. Right? And that’s the starting point. It’s let’s do personalization. 

Even once you’ve identified an audience, what does that mean? The way to solve that is to think about this from the audience’s perspective. So the goal here is the user’s goal. What is it that the audience is trying to do?

What is their mission on your website or your app? Why are they visiting? What is it that the weekly shoppers in my app want to do?

So I alluded to this earlier. I want to order the same stuff. Right? Because it’s toothpaste and bread and milk with ease.

Okay? So that’s one example. There’ll be other examples as well. And if you do a quick brainstorming exercise with your team, you know, you’d have a couple of ideas here. You’ll have a couple of theories and hypotheses again about what is it that this particular segment wants to do.

So you put your audience that you identified earlier and put it there. And then you brainstorm, what is it that they wanna do? You’ll have a couple of theories, and maybe those theories are informed by data. Maybe you’ve done usability testing. Maybe you’ve done customer interviews. Maybe you’ve analyzed your customer emails.

That’s great. You’re at an even better place, but it doesn’t have to be that. Once you’ve made this hypothesis, you’ve taken a stab at it, you can still go and validate it. Now you can go and look at, customer interviews.

And so let’s say you’ve got these 3, hypotheses here, and they’re quite varied. The next thing I would do before I build any test is I’ll do a couple of customer interviews.

What I normally say to people and they ask me, how many? I’ll say that one is better than 0. 3 is better than 1. 5 is better than 3. And you’ve done enough when the same themes start coming up, but, you know, you speak to a couple of users and you quickly get a sense of whether your theory is going in the right direction or not.

But let’s go with this one. They want to order the same stuff with ease. The next step then is to say, well, you may have come across this HMW (How Might We) do that? It’s quite a good question because it’s open.

We’re not committing to anything. Well, how would we do that? How might we, okay? So we can when returning app shoppers come back to the app, then we can show them previous orders very prominently. So that’s one of the things we can do. Maybe there are and again, you’ll have different ideas. So when you brainstorm this with your team, you’ll get a handful of ideas and the way to do this then is this becomes your variations. These become the things you actually start testing. And then the last one. This is where I’m not gonna spend too much time today, but it goes back to the framework that I’ve just shared with you.

You formulate that our office is based on everything you’ve got. You’ve had your theory. You’ve now gone and maybe you’ve spoken to customers. You’ve looked at GA, whatever you’ve done, but you’ve got some data behind it. You’ve identified your population already, your segment. You had a theory about what impact it would have.

So in this case, you know, that return behavior, that improved experience. But we’ve got to be clear about how we measure that, you know, how will we? What is the metric we’ll use, or what is the feedback we look for to know that we were right? And the rest of it is pretty similar to the way you would approach an A/B test. And that’s where I’m gonna leave it.

We’re almost out of time, but this is the framework. So let me remind you and let me summarize, and it’s really as simple as that. You start by identifying who it is that you’re going to, or who’s that audience. Don’t overcomplicate it. Don’t worry about minute segments. Don’t worry about expensive tools at this stage, or about hugely sophisticated data points.

Start with what you have and start by thinking in the context of your business. Who are those audiences? There are core audiences. We looked at some of them. What is it that they want to achieve?

Well, firstly, how big are they? And always go with the ones that have the bigger reach and the bigger impact, all other things being equal. What do those audiences want to do? How can you make their life better? What are they trying to get done in their life?

And how do you improve that? How do you enable them to do that? And then how, you know, how do you make that real? How do you bring that to life? And that becomes your personalized experience. And then the last thing I’ll say is, make sure you test it because for whatever reason, you get people who are very disciplined with A/B testing, with experimentation, and they test everything.

And then when it comes to personalization, they don’t for whatever reason. And I suspect, again, this is tools-driven because a lot of the tools I set up like that, don’t do that. You know, you’ll want to make sure that the experience you’re putting out there actually works. You’re not hurting sales, and, you wanna learn from it as well.

You want to, each time validate that theory, you build on it. You learn something that you can apply in a bigger context. Also, test the personalized experience against the entire audience. I’ve seen this many times that, that experience aimed at a subset of your total audience. When you test it against the entire audience, it actually applies to the entire audience as well.

And of course, you have a much wider reach when it’s sidewise. So, you’d rather do that. This vehicle is when I’m gonna where I’m gonna pause it and happy to take any questions or comments at the stage.

V:

Would encourage everyone to, you know, even if you have any observations and your opinions to share, feel free to, you know, drop a message on the questions box and we’ll be happy to take them up. Or maybe also you can request us to switch on your mic and you can speak over the audio directly, to Johann. So while the questions are coming in, while people decide on what they have to ask or what they have to share, gonna believe we can move forward and, I’ll let you know once there are inputs from the people.

JVT:

Yeah, Vipul. I can’t see the questions. If there are any, just alert me to it.

V:

Yeah. Absolutely. We do have one question that came in from Shane, let me just try unmuting you. And, you can ask your question with more context, to Johann.

So one minute and you can also unmute yourself from your side. We’ll be happy to hear more context around the question. Just let me know if you want me to read out your question. Cool. No problem.

So Shane is asking, “Is a low-volume website worth doing any personalization?

JVT:

Very good question, Shane. I would say, well, of course, you have to define low volume, but I would take it a step further, and I would say is low volume websites worth doing A/B testing. And, I mean, going back to I hope you can still see my screen, Shane, going back to this spectrum over here. I mean, this really with any website, even I work with websites that I’ve masses of data, huge retailers, even there, we start over here. This is where you’re gonna make the biggest impact.

And with many of those big retailers, it’s quite a while, a year longer even before we start stepping into some of these areas. it’s not quite like that. It’s in parallel, but certainly, I would encourage you whether it’s low traffic or high traffic to exploit the larger audiences. These two are on the left, here, as much as possible before you start really investing in the smaller audiences. don’t be scared to venture into it because you’ll learn that way as well.

But, with a small audience and, again, you have to define that, but with a small audience, I’d be, inclined to grow and optimize through A/B testing and, acquisition first and make the most of that audience before I start slicing it. There might be, in your specific, case, it’s really you’ve got to treat it on merit. So before I was with the agency, I worked with, one of my clients who was a relatively small, eCommerce business, not massive, traffic numbers. In that case, it made sense to personalize based on location, because it was clear in the numbers. There was such a distinction, such different behavior between the 2 dominant locations. So that’s the core we made there.

I hope that helps, Shane.

V:

Yeah. We hope that answers your question, Shane. Interesting question though. Yeah. I mean, Johann, is there any more part of the presentation left?

JVT:

No. No. That’s it, Vipul. I hope that was useful. 

V:

Because, we do have, you know, one interesting poll that we prepared for the audience. So I would love to get, an answer from the audience on this question. Let me, with your permission, Johann, let me just. A few people have dropped out already. I should have emailed to you earlier. Interesting to see.

So we’ll be very interested in knowing if your organization is currently doing any onset of personalization, you know, are you doing it? Yes, no, or would you have any plans to do it? Let us know. It’s all anonymous, so we won’t know.

Yeah, it’s a fifty-fifty person split between yes and plan to do it, but nobody has yet confirmed that they are not doing any on-site personalization at the moment. So just 10 seconds more, and I’ll close the poll. 3, 2, and 1. Great. So looked like 60% of the audience today, their organization is running an on-site personalization program that’s really great to hear because that’s the majority of the audience and 40% of the audience responded with plans to do it. So if you do, for those of you who plan to do it, I hope this was useful.

And if you seek help, if you liked the beautiful workshop that Johann created, may be able to guide you on how to take those first steps towards personalization, and we also help you set up personalization program at your organization. One question just dropped in from, Rajneesh. Rajneesh, would you like me to read out the question, or would you like to sit on your mic and ask the question yourself with more context?

Rajneesh:

Hi, Johann, and, Vipul. Thank you. So, because we were talking about the low-traffic websites, right, and especially around personalization there. One technique that is quite useful in such a scenario when we do CRO of maybe experience imagination or whatever you wanna call it, A/B testing. User research is used a lot where we actually talk to people and understand what their challenges are while completing a journey.

How do you see that working in the case of personalization? When you’re doing this, say, talking to 20 people, yeah, you cannot cover as many personas as you would like to do in case of personalization, but it may work for AB testing. So how do you see that working here?

JVT:

That’s such a good question, and thanks for asking it because this is quite an important topic actually on its own. So the way I’d like to think of it is validation right, validation ideas. And with A/B testing, we’re assuming that you’ve got enough samples. You’ve got a big enough audience as you alluded to as well to be able to run A/B tests.

And, of course, the smaller your audience, the smaller your website, you know, the less accessible may be the less relevant that is to you, but that’s not the only form of validation. It happens to be a very good form of validation, the best form of validation. But there are ways to validate concepts, theories, and hypotheses. If a, you don’t have enough of an audience, or, b, even if you do have an audience, you do have enough traffic, but you want to validate the concept before you get to the execution. Let me think of a quick example because we actually do a fair amount of this. So, the way you would wanna do that, the principle that I’d like to use at any point is at this time, when you’re talking about a particular idea or concept, what is the next thing I need to learn in order to move forward?

And what is the best way, the cheapest, and the quickest way to learn that thing? If the answer is an A/B test, great. But a few steps before that, maybe the answer is a couple of customer interviews. Maybe the answer is a survey, maybe the answer is, some prototypes that I put in front of users, customers. So something that I’ve done with a retail client who’s an omnichannel business that sort of has stores as well as the website and the app. We go into the store, and we approach shoppers in the store and we give them a wireframe. And often this is a hand-drawn wireframe. So it’s very rough. And, you know, the kind of things we’d ask them is, you know, where would you click? And what do you expect would happen next?

And, of course, we got a goal. You know, we give them a scenario, and they show us in this piece of paper. Okay. I click there, And then the question, what do you expect next? And, you know, you’re kind of, on just a very rough low phi prototype.

You get that immediate feedback. Based on that feedback, we then quickly drew another wireframe. We show it to other soft shoppers. Hey, you know, maybe we’ve got a sample of

20-30 more in a day. That’s great, but that is not about sample size with A/B testing. That scientific rigor, that statistical rigor is incredibly important. That’s the point of A/B thing. The point of these exercises is that early-stage validation is not statistically significant, and is not Bayesian probability.

It’s not that rigor. It’s about validating theories. It’s about validating assumptions. So any idea, any concept, any new concept, is based on assumptions. And what we’re really trying to do is we’re trying to find the quickest cheapest route to validating our assumptions and early on, it’s customer interviews.

It’s not about sample size. It’s about validating and moving us forward. And, you know, later on, when you get to the execution, it takes the form of an A/B test. I hope that makes sense.

R:

Thanks. Thanks, man. Yeah. It does. Yeah.

V:

Thanks, Rajesh. That was, indeed a really interesting question, because I never thought, about user research from that perspective. Thank you so much, Johann, for, sharing insights with the audience today on the topic of personalization. I really hope, and, I am confident as well that the audience will be walking away with, insights that they could, you know, quickly implement in their own organizations and get started with personalization.

If you have any more questions regarding the presentation or anything related to personalization or A/B testing in general, you know, you can also find Johann on social media. He is very active on LinkedIn as well. So feel free to reach out to him and he can give you tips and tricks on how to run a better A/B test and personalization program, maybe. So with that, we’ll just now wrap up the session. Thanks again, Johann, for a wonderful presentation.

Thank you.

JVT:

Thank you.

V:

I wish you and our audience a really good day ahead. Bye bye.

JVT:

Cheers everyone.