Do CRO the right way!

you who do not know what VWO is, VWO is a full-funnel website optimization, A/B testing, experimentation, and conversion rate optimization platform. Today, we have with us Nancy, senior CRO consultant at VWO. Nancy owns the strategy and execution of CRO programs for many of VWO’s clients across various industries ranging from e Commerce, Banking, Media, Hospitality, SAS, and much more. If you could please turn on your camera, Nancy, so that our audience could see you.

 

Nancy:

Hi, everyone.

 

Shanaz:

Thank you so much for doing this. I’m sure there will be a plethora of insights to take back from the session today. But before I pass on the mic to you, I’d like to thank everyone for tuning in from all over the world and inform you that we will be taking up questions at the end of the session. So please feel free to drop your questions at any given point during the presentation. With that, Nancy, the stage is all yours.

 

Nancy:

Thank you so much and ask for the introduction. In this webinar, I will be talking about CRO and the right way to do CRO. It’s a pretty “in” thing these days, since everybody’s talking about it. But in this session, I’ll try to not talk about stuff, about concepts, just theoretically, but give you something in which you can get an action.

Right? So let’s get started with that. What are we covering today? So, the first thing is the important reason behind no success from CRO for some businesses. Right?

Second, the thing is how to conduct research to generate data-driven high impacting hypotheses. 3rd is what’s the importance of prioritization framework. The 4th point is things to consider before implementing a test and lastly, what to do with your test results. Right? But before we start and go into the details of these points, I wanna quickly give a brief introduction about conversion rate optimization for people who are pretty much new to this.

So conversion rate optimization, to simply understand, is a systematic process of basically increasing the percentage of visitors who take a desired action while they interact with your business. So that action could be anything purchasing from your website, downloading an ebook, signing up for a free trial, signing up for a demo, etc. Right? So you’re trying to have a goal for your website. You’re trying to increase the conversion rate for that goal by optimizing the conversion funnel. 

That is simply conversion rate optimization. Right. So now that you have an understanding of conversion rate optimization, let’s get into the details of the agenda that we have for the session. The first is the important reason behind no success from CRO for some businesses. Now, a lot of businesses aspire to do CRO or like Netflix, Airbnb, Spotify, etc. because they’ve had massive success from experimentation, right?

But that rarely happens when companies start doing CRO experimentation, and the reason for that is there is no structured process, and it’s haphazard testing. So people are pretty much enthusiastic; they’re excited. They have a test list of test ideas they want to execute, but what they lack is proper process and detailed planning.

They will not have a clear goal, you know, on which they wanna focus. They will not do a lot of research, so their hypothesis will not have data points which will sort of validate or, you know, support that hypothesis. They will not even define the hypothesis. So they will just have a change in their mind which they want to test, but they don’t clearly know if they implement that test, what they are expecting and why they are expecting it? What are the assumptions and what is the data that is backing the assumption? They want to execute, but what they lack is proper process and detailed planning.

Right? They do not have any clear testing roadmap in mind, and they’re skipping steps which are leading to poor quality. They have to re-run the test because they skipped some of the steps. So all of this, because there is no structured process, there is no framework, it leads to tests that provide no uplift and no learning. Right?

So the question comes, what is the right way to do CRO? So it is data-backed iterative testing, right? So, you must have come across a lot of frameworks and fancy words about conversion rate optimization processes and steps on the internet.

But it’s fairly straightforward, fairly simple, and what you see on the slide, you’ll be able to understand. So the first thing is you need to define the goal and KPIs – Key Performance Indicators – that’ll basically help you track the performance and impact the goal that you have in mind. Now, once you have the direction defined, you start digging deeper into what problems are there which are sort of impacting your growth, impacting your conversion rate. So you collect data, you analyze data to understand the struggles of your users, the problems they’re facing, right?

Next, you move on to a solution, and you try to find out what are the solutions that will help you fix those leaks, right? Reduce restrictions on your website. And once you have those solutions, then you put it into testing. And lastly is deriving learnings. That is post-testing analysis.

So once you have run the test, you have stopped the test. You will sort of analyze whether it, you know, was a winner. It gave you an uplift or it didn’t give you an uplift or how your user behavior got impacted, how the KPIs, the goals got impacted with the changes that you introduced. Right? And every time you conclude a test, you have a new learning, which again goes back into that loop where you are able to discover new problems that your users are facing.

You find a solution for it. You go for testing out that solution and that goes on. Right? So it’s an iterative process, but data-backed, right? Companies that do not do much planning do not focus a lot on the data part. They usually already have assumptions in their mind. Based on those intuitions, they will come up with test studies, and they want to execute those test studies, which is why they do not see a lot of success. Right? 

But if you follow this process, these steps, you will 100% have uplift in your revenue plus learnings about your user behavior that you can apply in other functions like sales, marketing, and other client-facing teams can also use those learnings, right?

So that is about the right way to do CRO. Next is how to conduct research to generate data-driven high-impact hypotheses. Now, we are focusing a lot on, you know, data and having data-backed testing ideas. So it’s important that we talk about it in detail, right?

So this is the section that covers the details about how to conduct research. The first thing that you need to do is set up the goal. You need to write down your business goals that might be overarching long-term goals. You know, maybe what you want to achieve in 3 or 5 years. Next, once you have that, you cascade your business goals into conversion goals.

Right? So for instance, you can ask how your website contributes towards your business goal? How does your mobile app contribute towards your conversion goal, right? So once you have these conversion goals, for every website conversion goal, there will be a set of metrics.

Right? And these metrics will help you track the performance of your campaigns plus the performance of different sections of the website. And once you have defined these metrics, now you can start working on the ideas.

You can start working on, you know, introducing changes to improve the performance of your website. So that is the 1st step. Another thing in goal setting, which is worth mentioning here – macro conversions and micro conversions. So macro conversion, as the name suggests, is about the top-line goal, the primary objective of your website. So it is usually revenue-based or lead acquisition-based.

For an eCommerce website, probably, it’ll be, of course, purchasing stuff from the website, right? Micro conversion, on the other hand, is more engagement-based, navigation-based, right? So for instance, you download an ebook from the website or you watch some video, some product demo video you watch online, right? 

So those are micro conversions. Micro conversions show that you are progressing. You are able to get engagement from the user. And eventually, he will end up converting for the macro goal, but it does not really guarantee that if a person, let’s say he watches for a demo, he will definitely sign up for maybe a trial from your website, right? 

But it’s very important that you understand not all the campaigns can be judged on the basis of macro conversion. For instance, for an eCommerce website, if you’re running a campaign on collection pages or category pages, you cannot judge whether it’s a winning campaign or a losing campaign on the basis of purchase goal because it’s way far from that page. And in between there are multiple pages, multiple information, multiple content, a lot of content that is shown to the user that will influence his decision, right? So in that case, the success defining metric cannot be a purchase goal, but rather these engagement goals like, people visiting the product details page and doing act to cart on product details page. That would be a better metric to judge the performance of a campaign that you’re running on the collection page, right? 

So, that is about the fourth step that you need to have clarity about the goals. Next comes benchmarking. So, this means you need to understand where you stand in the industry?

What is the performance of your 25% companies in your industry? And where are you? Like, are you doing better? Are you doing worse, or are you in the same position? So you need to ask questions like, what is the conversion rate of add-to-cart for top 25% businesses and now see what is yours, right?

So, that will help you understand how much improvement you need to make. It’ll help you again doing that goal setting, how much you can really achieve. Right? For instance, I can give you an example of, for instance, this eCommerce industry benchmarking says that top 25% companies have a card abandonment rate of less than 30%.

Now if your analysis shows that your cart abandonment rate is higher than 70%, so benchmarking exercise is enabling you to sort of prioritize the cart page, make changes there, fix the leaks so that you’re able to impact your conversion rate. Right? So, that is why benchmarking is important to understand where you stand, and you can set goals for yourself. The next step is collecting and analyzing data.

Now you need to identify problems. You can identify problems by collecting data. Right? There are multiple sources from which you can collect this data. So, the first thing you need to do is you need to understand your buyer persona. Second is observing data.

So, this means quantitative analysis. Third is observing behavior. That means qualitative analysis. Fourth is observing opinion. That means not just looking at the data, but actually looking at what people are saying and maybe talking to them, right?

Last is competitor analysis. So, let’s have a look at all of these points individually, how you can collect data. First is understanding your persona. So, you have set a goal, you know, for yourself, how do you, you know, how do you plan to sort of get the ROI from your CRO efforts, but it’s important that you identify the buyer persona that will help you achieve those goals. So, you need to have a clear understanding in your mind, about your target user persona.

So, you need to have a detailed understanding of his age, his location, his income, education, what are his hobbies, what device does he use? Is he single? Is he married? Is he married with kids? Every detail you need to have what are the information sources that he used to make a decision, right?

And this kind of information you can easily find from you know, these social media analytics reports, Google analytics, all of these, you know, sources you can use in order to get this information. As a result of this, what you will be able to do is, basically understand the copy and the design that you need to have on your website that’ll suit your target user persona. For instance, if you are an eCommerce website that sells luxury handbags to women and you see that a lot of traffic that you see on your website coming is from females within the age bracket of 18 to 24. However, your conversion rate is how you are for the audience, which is in the age bracket of 35 plus.

So, you know the content that you’re gonna show on the website, the design that you’re gonna show, the sentiment, the tonality of the copy, it all needs to suit the requirements of this 35 plus segment of audience, right? So that is the importance of understanding buyer persona. Next is doing quantitative analysis. So Google Analytics is one of the most common and popular tools that, you know, businesses use to understand what are the worst performing pages, what are the top performing pages, and where exactly is the funnel broken, right?

Because that will tell you what is the problem, and then you can move on to a real understanding that why are our people not moving forward, why are they dropping off from that stage? So for instance, taking that previous example of a cart page, if you see that 70% of people drop off from the cart page, you have to work on the cart page to increase basically cart-to-checkout rates. You have to push more people towards the checkout so that they complete the purchase and when they’re dropping off from the cart page.

So this will tell you where your funnel is broken. But the answer to the question of why people are leaving from that page, you will get from qualitative analysis. And that can happen by looking at the browsing patterns of visitors on your website. You can look at session recordings, heatmaps, click maps, and scroll maps. All of these tools you can use in order to understand how people are interacting with the web page, where you see the drop off happening.

Where are the places they are pausing and where do they sort of are clicking and they’re not clicking, all of those things you will understand, right? So for instance, on the cart page, you are able to see that only 10% clicks are happening on the “Go-to checkout” button on the cart page that takes users to the checkout page. Right?

And scroll maps reveal that at the bottom section of the page, where the checkout button is located, only 20% people are scrolling up till that section of the cart page. So you know now that because of the visibility issue, the  “Go-to checkout” button clicks are very low, right? So this will help you understand why people are dropping off.

So an additional way you can also figure out what the problems are is doing heuristic review. That means you sort of try to put yourself into the shoes of the end user, try to empathize with them, and see what struggles they are going through, what information is lacking on the website, which is making decision-making a little tough for them, and which is why they’re not confident enough. 

So qualitative analysis using heatmap session recordings, plus doing heuristic review, will further reveal insights about why people are not moving forward. What are their anxiety points? What are their doubts that they are going through, right? Also, another thing that I would like to mention is that whenever you come across any observation, it’s important to ask enough “whys,” right? And that’ll enable you to basically form a solid hypothesis. So for instance, in the case of this cart page example, you come across this data point that 70% of people are dropping off from the cart page. 

You ask, why is that happening? You get an answer because only 10% of people click on the “Go to checkout” button that takes them to the next page. Why is that? Because people are not able to see that button. Why are they not able to see that button? Because it is present at the bottom of the page where only 20% of people reach, right? So this will help you come closer to the change that you should execute to have a higher impact and have an uplift and fix that.

So that is about qualitative analysis. Next is observing opinions. Now whatever you have done, up till now, it’s all about you seeing the data and you are interpreting it. Right? And, you know, information can be interpreted in multiple ways by, you know, different people.

So to be more accurate, You can also talk to your visitors, talk to your customers, look at what they are writing on websites, look at what they are writing on social media websites, in the comment section, all of that. Right? So you are directly understanding what they are saying, about your business, about your products, about your services. For surveys and polls, there’s one thing that I would like to mention. It’s very important that you ask the right question to the right type of audience at the right time.

This means that you cannot ask questions to all types of visitors. You need to differentiate site visitors from site customers because customers will have a different awareness level. They will have more context, more knowledge about your product and services, while the site visitors will not have as much information. So site visitors can use this segment to ask questions that reveal the pain points, the anxiety points. 

For site customers, you can ask questions that reveal the USP, the moment so that you’re able to use that information and highlight it on your home page, on your landing page, your US page, because a lot of time it happens that what you think is your USP is not exactly the same as what people think your customers think as your USP, right? So this is how you will be able to get a lot of understanding of test ideas that you can really execute on the website. 

Next is competitor analysis. So, it’s very important that you are not repeating the mistakes that your competitors have already made. So you can avoid those mistakes.

You can look at the top 25% performers and bottom 25% performers in your industry, what has worked for them, what mistakes they have made and accordingly make decisions, and make changes on your website. So this will save a lot of time and effort for you. Because you won’t be repeating that mistake. And whatever has worked for these guys, you can take inspiration from that, right?

So that is all about collecting data and analyzing data. Next comes the hypothesis. So once you have collected data and analyzed data, you will have solutions in your mind, right? Now at this stage, it is very important to note that you need not just have a change in mind which you are just blindly following and putting it to execution.

You need to define it in the format of a hypothesis. That means there is a dependent variable, there is an independent variable, and you have an assumption which is backed by data. And this is the format that you can use if, for instance, in our eCommerce cart page example. If I place the “Go to checkout” button in the first hold on the cart page, it will result in more clicks on the button in the cart page. Because, right now, the visibility of the “Go to checkout” button is very low since it is present at the bottom of the cart page.

So this format really helps you think through your ideas. Right? And, it also clarifies the goals that you want to really track and define what primary metric or success-defining metric you will be keeping for the campaign. Right? So that is about creating hypotheses.

Next comes the prioritization. Now, a lot of times, it’ll happen that for a problem that you’ve identified, you will have multiple solutions. Right? And in that case, you need to decide which solution you want to test out first. Now in that case, if you do not have any process, if you do not have any framework in your mind, maybe the highest person in the organization or your manager might tell you that, you know, based on his assumption that test out this particular idea first.

Right? So that is not going to work out. That will not again give you a lot of results in terms of uplift or learning. So you need to have a prioritization framework because it’ll help you eliminate that bias, subjectivity, and prevent aimless testing. You will have more planning.

You will be judging each and every campaign on the basis of the impact, effort, and these different parameters that we have in prioritization frameworks. So the prioritization framework that VWO uses is the ICE model. There are multiple others that you can find on the internet, but this is the one that we use. So “I” stands for impact. “C” stands for confidence, and “E” stands for ease.

So “I” is a projected impact. And that you can judge by looking at the location of the test. How many people are you going to impact? How many people are you going to reach for that change? Is the audience of, you know, if you get very costly traffic on those set of target URLs that is, again, going to be very important and high impacting.

“C” stands for confidence. So you can judge in terms of how many data points you have to support your hypothesis. If there is one idea which is supported by just one data point, you will score it low on 1 to 5 scale. But if there is another idea which is supported by 5 data points, it’ll have a higher score. Or, you have case studies, probably, that test idea has worked for some other business in the same industry. And that will also increase your confidence that it may work for your business as well.

So you need to gather evidence from the past to build this confidence and rate your idea on a scale of 1 to 5 in terms of how much confidence you have. Lastly is effort. So effort is all about how much time, number of days, and resources would be spent on developing that campaign: on doing copywriting, designing, doing Q&A, all of that stuff. Right? So execution-wise, all things are required.

If that effort is going to be high, then, you know, you will of course probably de-prioritize, and the ones which will be easy to execute and will have a higher impact – you would want to prioritize them first. So, right, that is what I wanted to highlight in prioritization.

Next, we have things to consider before implementing a test. So this is a step which is highly underestimated by a lot of people. Once they have a test idea in their mind, they have prioritized the test idea, they move on to just doing the execution. But before doing the execution, before running the test, it’s very important that you spend some time understanding the type of test. You know you will use it in order to execute the test, whether it’ll be an A/B test, split test, or an MVT. What are the goals that you will have sort of tracking? And you need to define which will be a success-defining metric, which we call as the primary metric, that captures the true performance, the true impact of that change. You need to define your secondary goals. How people would be moving forward in the sort of way they are moving forward in the journey. Right? So you are able to understand that if they move forward from step 1, what do they do on step 3 on step 4 and so on? And then, guardrail metrics will be mostly your macro goal. For instance, in the case of an eCommerce business, if you are doing a change on the collection page, and you have defined add-to-cart on the product page as your primary metric, you do not want your purchases to get negatively impacted while you are pursuing this add-to-cart goal.

So guardrail metrics usually are the ones which you do not want to have any negative impact. Maybe they might have no impact or positive impact. Right? So the focus will be on primary.

And lastly is the configuration. So a lot of people again do not think through what pages they wanna target or audience they wanna target and they miss on integrations with other reporting tools that are there like Google Analytics. So it’s very important that you’re spending time to understand and decide, your goals, the type of test you want to choose for instance, in this case, A/B testing. So A/B testing, you can use when there are just 2 to 4 variables, 2 to 4 changes involved. But if you see that you are doing a complete revamp and there are a lot of, you know, variables involved, you can go for a split test. Right?

So It will be a completely new page, completely new URL, and that is how you can execute that test. Or you can go for a multi year test, which we call as MBT, when you want to actually see all the combinations of, you know, multiple changes that you’ve introduced. Right? So These all things come under the design of an experiment, which is very important, to decide before you start running a test. 

Lastly, we have what to do with your test results. So this means that you have, you know, run your test and now you have paused your test. You are concluding the test, right? So there’ll be 2 scenarios. Either you will have an uplift or you will see that variation underperformed.

Your control was still there. So what do you need to do in both these scenarios? When you have an uplift, you need to see whether you have, you know, whatever, like, number you had in mind. For instance, you were targeting 10% uplift, but you got 5% uplift. So you need to see how much uplift you have got against what was your target.

And also analyze the cost of deploying the changes and whether that justifies the uplift that you’ve got is enough to cover the cost of deploying the changes, right? So that is something that you need to decide before shipping the changes once you get a winning variation. On the other hand, when you see that variation is not performing well, in this case, you do not reject the, you know, campaign altogether and just you know, declare it a failure, a failed variation. You, in fact, go on to further, you know, segmenting your reports to see if there are any small wins.

Maybe not on the whole, but for, you know, desktop new users, it might have worked out well, right? So you need to do segmentation to reveal further insight. And also, it’s equally important to look into the recordings and heatmaps and scroll maps and do that qualitative analysis again. To understand how the audience reacted to the changes, how their behavior actually changed.

Because just looking at the number that it gave an uplift, it didn’t give an uplift is not enough. You need to answer the why behind all of that, right? And why, the answer to that question, you will only get after you do qualitative analysis using user recordings, heatmaps, full maps, etcetera, right?

And as I said, once you conclude the campaign, you will always have something new to learn, which you can again, you know, sort of use to reconstruct a new hypothesis and, you know, run a new test again, right? So that is how it works when you have a variation which is not performing. Now, there are certain things that you need to do, irrespective of whatever is your result, whether it’s a winning variation or not. First, you have to define the conclusion.

You have to state whether it’s a winning variation or whether the control is better, right? Next, you need to document business learnings. It’s very important that you maintain your document, all of these learnings that you’re getting so that you can sort of use that learning and again, for another page around the section of the website, you can reuse that. And you’re not making the same mistake again.

The same goes for operational learnings. You need to document operational learning. So in case you faced some challenges while configuring the test, whether it was related to the audience or goal configuration – so all of those challenges that you faced and then you found out the solution for it, you can document it so that you can save your time when in the future you are running a similar test. You are using a similar audience for the test.

So it’s very important that whatever challenges you face, whatever operational learnings you have, you document. And lastly, sharing and evangelizing the learnings because you can have real success from conversion rate optimization efforts when you are sharing the learnings with other teams, right? Because in that case, you are able to pass on the learnings and they are able to actually use that learning and make their decision accordingly. They are not making the same mistake.

There is some learning that you passed on to them. So they learn from it, and they are also improving. So those learnings could be used in your other functions so that we’re able to make better marketing decisions for sales decisions and in other functions as well.

So that is about the last step which is concluding a test.

And then with that, we come to the end of the session. So if you have any questions, I’ll be happy to take them up and answer your questions

 

Shanaz:

Thank you, Nancy. That was really insightful. We do have a lot of questions. Let me just address some of them. So we will be sharing the recording as well as the slides, with all the attendees and all the registrants within 24 to 48 hours from now.

So whoever missed the session in the beginning or whoever wasn’t able to attend, there’s nothing to worry about. They’ll reach you straight to your inboxes. So with that, let me take the first question. I’m not sure if I’m pronouncing your name correctly.

Pardon me if I’m not. Vyaches Laf is asking if VWO can be used both for client-side and server-side testing.

 

Nancy:

Yes. 100%. So, we do have a VWO FullStack product that you can use for server-side testing. If you go to the vwo.com website, you will be able to see in the product section. It’s listed there.

You can easily find out the details about that. So, yeah, we do have this product.

 

Shanaz:

Next question is by Alexis Martin. Alexis is asking if there are any tips for not getting overwhelmed when looking at heatmaps and session recordings. Often I don’t see a clear pattern in terms of user behavior.

 

Nancy

Okay. Okay. So, it’s very important firstly, that you are looking at how much sample size you have, you know, at times, you know, this is the challenge which usually comes when you have very less number of clicks. You are just looking at probably 200, 300 clicks, and they are not sufficient enough, and you don’t see a lot of interaction, right? So first thing, ensure that you have enough sample size. You have enough data to understand and gain valuable learnings from heatmaps.

Another thing is if you’re not able to understand heatmaps, look at click maps because that will give you a percentage like 10% of people clicked on X, Y, Z element. So you will have more specific information in terms of engagement when you look at the click map. Maybe a heatmap, because it works on the color basis, you’re not able to understand. So use click maps in that case.

And scroll maps will help you a lot in understanding till what depth your audience is, you know, reading your page, right? For instance, you see that there is some information which is present at 3rd fold and people who do interact with that 3rd fold who do, you know, click there, they have a higher conversion rate. So you know that, you know, right now, only 30% of people are able to see that section because your scroll map shows that. You can now prioritize whatever information is given in that section on the first fold or the second fold where you have, you know, more visibility, right?

So I think that will help you a lot because scroll maps plus click maps give you more sort of in terms of percentages, the engagement data.

 

Shanaz:

Yeah. I hope that answers your question, Alexis. So the next question is from Renee Selza. Renee is asking if it is better to isolate pieces or test whole new versions altogether.

 

Nancy:

Okay. Well, see, this depends on your hypothesis. If your changes are related and they are sort of forming one hypothesis, then you can have those multiple changes tested in one test campaign, right? And you can have maybe multiple variations.

In one variation, you will just have 2 changes; in the other variation, you have the other 2 changes, right? But if you have a lot of changes, maybe 8 to 10 changes that are involved in your hypothesis, it’s better to do a split test in that case rather than doing variations because that might divide your entire traffic, and it’ll take a lot more time to conclude this step, right?

But, yeah, it all depends on your hypothesis, how your hypothesis is. If your hypothesis involves that all of those changes are going to work for a common goal and the data is also supporting that, you can have one campaign with multiple variations as a test campaign for such an idea.

 

Shanaz:

Yeah. I think data is the answer to everything. Based on the data you have and the goals you’ve set for that particular campaign that should tell you, how many elements or do you need, like, complete design or stuff like that? Alright. I hope that answered your question. The next question is from Supriya Agarwal – How should insights from customer surveys be embedded into a CRO experiment? And if surveys are not responded to, how do you get relevant insights?

 

Nancy:

Okay. To get conversion, like, to get engagement and service, it’s very important you’re asking the right type of questions at the right time, right?

Timing is very important. You cannot ask a question when a person is probably, you know, doing something else and you are, you know, throwing the survey on their face, which is sort of distracting them and it is quite intrusive, right? So you need to have the timing. You know, you need to do research, you know, whatever your question, whatever is accordingly do some research, how other businesses have, you know, at what triggers they have, you know, at what time and what triggers they have used in order to show the survey.

Another thing is, you know, to the right audience, you are asking the right questions. So as I mentioned, a site visitor is not going to give you a detailed answer because he does not have a lot of information plus he’s not that much invested in you, right? So you can only have multiple-choice questions for such a visitor. But a person who has already purchased from you, maybe he will be open to giving you more detailed feedback, right? 

And how does this go into your hypothesis library? As I mentioned, for instance, you are trying to understand the USPs that you should highlight on your homepage so that you’re able to increase the engagement of the audience of the new visitors from the home page to your, you know, other, bottom of the funnel pages. So in that case, you can ask your site converters. We’ve already purchased the top 3 things that they loved about you, loved about the product, about the service, and everything.

And this insight will tell you what it is exactly that is giving them that ‘Aha’ moment or it’s a great thing for them which is working out, and you can use that information to add those UX fields on the home page. Instead of having generic ones which people use like free shipping and everything, you can have real USPs, which your visitors, your converters are actually telling you, right? That is the reason why they purchased it.

 

Shanaz:

Yep. So the next question is by Rodrigo. Rodrigo, you’re asking if we have any real CRO examples to show. So, for that, I would suggest, head on to vwo.com/resources, and you’ll see more than a 100 case study success stories that companies have shared with us using VWO. The successes that organizations have experienced using VWO and doing CRO. So, yeah, I think you’ll get a lot of data for a lot of industries and an endless type of campaigns that people have run and seen successes.

Yeah. Sure. So, Sumit Ramesh is asking that most tools do not show the cart page heatmap or scroll map since the cart page keeps on changing. Is there a way to set up behavior reports on cart pages?

 

Nancy:

Well, I think for vwo.com, if you use heatmaps there, you should be able to see the cart page. And no matter, like, I would say that, you know, whatever changes you’re doing, you will be able to see that updated heatmap in whatever tool you are using. Right? So I don’t see a problem in that. I think that that is more about the tool that you’re using.

 

Shanaz:

Yep. So the next question is by Tee Buyi. He’s asking for suggestions. Buyi’s company is working on revamping the site based on personal opinions, not related to testing. But they want him to conduct tests at the same time when he has no idea what the new design is. So how should he approach this?

 

Nancy:

Okay. Well, when your company has decided to do a revamp, to implement a new design, I would say it’s not worth testing, right now, maybe because you know that your efforts will not really be valuable, as they have already decided on something new. In this case, I would rather suggest you speak to your leadership who have planned this change, who are deciding this change. And tell them that let’s test that new design in sections, and element-wise you can test, right?

For instance, you can pick one page, and you can make some changes in the new design so that you are able to gather evidence whether that new design is going to work or not. And you will actually save them from losses because if they go for a complete revamp without testing, they would not know whether it’s working out well or not. And most of the time because of the new design, the site visitors, they’re not used to the new design. They will find a problem.

They will struggle, moving forward, and they will not really convert. So instead of doing that all-in-on change, I would suggest doing incremental sort of testing. Whatever design you have, the new design, you have to pick bits and pieces. Test them out. Validate them and then move forward towards doing the revamp.

Once you have tested and you know that this is going to work out. And if it does not, then you need to again suggest to them that this is the modification that should be done in the new design.

 

Shanaz:

Yep. I hope that answers your question. So, we’re running out of time. There are a lot of questions. So, I’m going to scroll through and see how many of these…

So Shay Hackham is asking, how many recordings will be enough for pointing out a problem among thousands of recordings, or is it only a verification for a problem you already noticed?

 

Nancy:

Yeah. So you need not see recordings of the entire website. You already have identified that this is the page and this is the segment of the audience for which the problem is, for instance, your new visitors on the product page; they’re not doing add-to-cart. The conversion rate is very low. 

So you will only pick that audience and you will view recordings for them to understand why.

Somewhere you will have an idea, maybe 30%, 40%, by doing a heuristic review, but you will get more confident in your assumption when you see that users, what they are doing. You know, when you see the recording. So I would say, not less than 200 recordings you should see.

Definitely, 200 is something you should always look at.

 

Shanaz:

Yes, I hope that answers your questions, Shay Hackham. With that, we’d like to wrap up this session as we’re running out of time.

There are a lot of questions, but please reach out to Nancy or me on LinkedIn if you want to continue this conversation or if you want to know anything else regarding CRO or VWO. We’d be happy to help you with all of those answers and clearing your doubts from the session or otherwise.

Thank you so much, Nancy, for joining us. It was an awesome session with a lot of insights. I can see that by the number of questions that are pouring in even now.

Thank you so much for doing this. And thank you everyone who attended the session. We look forward to seeing you in our next webinar. Alright. Thank you, Nancy.