VWO Demo for Google Optimize Users

And, the time is of Google Optimize sunset when a lot of people are looking for a new solution for their ongoing experimentation activities. They are discovering new questions around all this. And, that is why, I’m happy to work with two guests here who will give a lot of answers to the incoming questions. So, welcome, David Otero. David is a former colleague of mine.

He’s a very good friend of mine, and, he is probably, from the bottom of my heart, the one who knows most about conversion rate optimization, I think everything I know about it, I learned it from you, David, many years back. And so thank you for joining us here, and, for being with us to answer a lot of questions and to walk through the secret sauce of conversion rate optimization with VWO.

David Otero:

Thank you for having me, Jan. A pleasure.

Jan:

And, a few hours ago, I thought somebody’s missing here. Originally, it was planned that David and I would run the show and so on, but we have somebody on the team who is really providing excellent live demos and has the capacity to really, boil it down to the functions that really matter. So welcome Sergio to our session here, and thank you very much, for making yourself available on such short notice.

Sergio Pardo:

Mhmm. No problem at all. And we have the pleasure to be here. The US.

Jan:

So what’s the purpose of this meeting here, and, why are we all here? And, what is it that you can expect from us today? Let me wrap this up. A couple of months back, Google announced that they would sunset Google Optimize. That means that 100s, if not 1000s of companies around the globe are now looking for their next solution.

But we noticed that it’s even more than just a migration from this tool to another tool. It is actually that a lot of companies and teams are asking themselves how are we using it? And, what is it that we actually need? So what is it, and how can we deploy our future tool in a way that we create a more effective conversion rate optimization program? I think that’s the much more interesting question here rather than the question of, okay, what is the software that we’re gonna use?

And, so the idea is that we’re gonna walk through the entire process of optimization and experimentation from the very beginning to the end. And then at specific moments, give you an idea of, how VWO is able to deal with a related task and, how we sort this. And this is when Sergio will give us a sneak preview of the particular functions. Before we jump right into it, I would like to, invite the audience, and encourage the audience to really ask questions. It doesn’t happen so often that you have David with us here.

I mean, you can also ask him, at other occasions, but, now, in life and in color, you could share your questions in the particular question section. And my colleague, people in the back, he will, let me know when the first questions come in, and then we’re happy to jump in. You can ask a question at any time. Sometimes we’re gonna answer them a little bit later, and sometimes, we’re gonna answer them straight away. Right. So let’s get into it.

As I said, we didn’t want to just run a live demo on VWO. We wanted to look at the entire process of optimization. And, yeah, David, you and I, we’ve walked through this process quite often. Sometimes it was a painful process. Sometimes, it took quite a while.

And so on, a question for you. What is the best way to get started if you build or scale your optimization program?

David:

Yeah. That’s a good question. But also the thing that you mentioned is having a process in place, it’s actually important. So you’re just throwing the process work, but it’s super important. And I would say that the beginning of everything is analyzing.

So we need to understand. We need to and we live today in a data-driven world. We are flooded with data here and there. But sometimes we just go and don’t look at it. That’s why I would say what I would say.

And based on our experience together with you and with, a lot of clients, we’ve seen that the first part, that is, basic is analyzing the data to identify issues and or opportunities or converting issues into opportunities. So, yeah, that’s what I think is crucial. And I would say oh, no. I’ll go for it. Sorry.

Jan:

I’m sorry. I mean, I’m always surprised because, since, let’s say many, many years, particularly old, serious website owners run Google Analytics or any other type of analytics solution on it. Would you say that companies make sufficient use of all the information that is, that analytics offers them?

David:

I don’t think so. I mean, there’s companies that, yes, of course, they have departments of data with analysts, that a scientist and one of, of course. But I would say that the vast majority of companies, just use a basic GA implementation and do not dig deep. They might notice some things or they just look at transactions, how much money am I making, or how many sales but they don’t go and start looking at data in a different way. And, also, what I wanted to go to is that there are other options out there, not just data from an analytics solution. And you at VWO have a session recording and heat maps, for example, which is, an assisted way of understanding how users are interacting with the website.

Jan:

Yeah. Right. I mean, dear people in the audience, I didn’t pay David to say this. He honestly means it. And it’s true. There is a quantitative data. There’s a lot of information hidden in the data of Google Analytics, but there is stuff that is not there. It’s qualitative UX analytics. Sergio, maybe you could give us a quick overview of what is included in VWO Insights. That’s the toolbox David mentioned here.

So could you give us a short sneak preview of what’s included in VWO Insights?

Sergio

Yeah. Definitely. Well, I think you’re seeing my screen now. Actually, the internet is one of the models we have inside the VWO suite. As Jan was saying, the qualitative data is what really helps us understand where is the problem, right?

Why is this not happening? How I would like this to happen. Right? So inside Insights, we have 6 different tools. All of these tools, are, as I said, you know, it’s a way to understand and also create some ideas for them later, maybe test, organize your optimization roadmap, right. So, of course, we have, the usual, predictive, tools such as heat maps, where you can see how people behave on your website, and session recordings of individual users.

Also, you have GA4 analytics, which is a very, very powerful tool to actually a lot for analytics. You can see even, specific information, very detailed information about what people do with a specific, field in a form, you know, and you can work on that. And also, you have, of course, funnels. You can build that user journey that is ideal for you and see where are people dropping out, how can I modify something to make people go further in the funnel? You know, so everything is that way.

Right? So finding the reasons.

Jan:

Thanks. I think from these 6 functionalities that you mentioned, the most, the least known ones are the form analysis that you just mentioned, and we’ve seen really amazing stuff there to better understand your checkout forms or your registration forms. Anybody who is acquiring leads on the website where people drop off while they are filling out a form. So there’s a lot of information in that.

It’s not that well known like other tools. And, as I always, point out when we run the full live demo, Superbase is also the last bullet, which is extremely helpful. You don’t need 100s of 1000s of replies to your questionnaire. Sometimes you need 10. And then you have really good input for the optimization process. So that’s a good point. Analysis. Thank you, Sergio. David, when everything’s analyzed, this triggers a lot of ideas for optimization. Right? And how should people address the challenge of then prioritizing the right ideas and putting them into the right sequence of their optimization process?

David:

So I would say first, you need to come up with the idea, of course, and that comes with a hypothesis. So before even prioritizing, you need to come up with the hypothesis, which is usually in the form of if I do whatever we do, then I’m gonna get whatever we want. Right?

So if I change this CTA, I’m gonna then have more transactions, for example. So writing or defining the hypothesis for each idea is really important because it helps bring down our crazy ideas. Also, something that’s really, really important is to make sure that it’s, our ideas slash hypothesis, that can be done because sometimes we have wild ideas and wild dreams. I would like to change the entire funnel.

Okay. So we’re testing. I mean, maybe it doesn’t make sense to change the entire thing, or it’s not technically feasible, or we cannot measure success. If we change that entire page, how do we know what really drove the test? So those things, I would say, come first before actually prioritizing the ideas once we have them down.

Jan:

Yeah. I think that’s a good point. I mean, we’ve got something in our application, which is not a product. It comes automatically through, which is called the VWO Plan. And I’m always surprised that not everybody knows that, but in the VWO Plan, we offer a very simple template to actually formulate a proper hypothesis.

Maybe it’s okay. You could give us an idea. What does that look like? How do you create a hypothesis in the VWO Plan?

Sergio:

It’s very, very easy. Actually, you know, you can see this, this kind of, dashboard we have here. We have one card for every hypothesis we have. And the very interesting thing is, that, you know, as David said, we expect that by reducing the number of elements in a bar, for example, we will address more clicks in the remaining elements. Right?

The very interesting thing about planning with the creation of the hypothesis is that we always need to know which one I’m going to address first, right, which one is the one who is going to impact more, the goals I have in my website and in my business. So that would be very useful. That framework we use in CRO, the ice framework, where we can give confidence, maybe I’m very confident this hypothesis is going to work. I would give it a 5. Maybe it’s something very important because we have a lot of traffic on the home page or we’re working in the low part of the funnel in the checkout part.

That would be a 5, of course, if we’re working with, a hypothesis that would be on another low-traffic page, we would keep it a 1. And then, of course, there is implementation. If this is something that we can change, maybe with the visual editor of VWO, which allows you to do a lot of things, we’ll give it a 5. Right? And with that, we don’t have to create that.

David:

Let me transfer you here because this is a common misconception of the IS scoring. If something is easy to do, you have to lower the score. So something super easy is a 1, and something complicated is a 5. And that is because when you add the numbers, you can’t pride price. Sorry.

Please.

Sergio:

Thank you for the point then, actually.

Okay. So that would be the first one. The Plan is very useful to create that hypothesis and prioritize between them. You know, you can see the scoring you have there, and you can compare it with the other scoring, and you can see easily and fast which one you need to address first. The other interesting thing about, the Plan is observation.

Observation is kind of a wall, you know, a place where you can just leave there all the ideas you have regarding your CRO program. And if you’re in any website, your website or any website you’re navigating, you can just activate it with a click, have some observations here, and say, okay, I like this now, this number, for example. Right? And we would have, 1st, and this, observation created in our platform, and also we would have a screenshot of this website to remember what this would be really like.And we have all the information we are gathering about the ideas in the same place.

That is very interesting.

Jan:

Yeah. Thanks. I think it’s a really helpful tool. It’s that there are many other ways, and I know they did that in a lot of agencies like making science and others. You have your own, status tracker, deeper task management with it, and so on.

I don’t think that VWO Plan is competing with that, but it just gives you a very quick simple overview of all the experiments that are running. And given the fact that it also includes the option to optimize, let’s say, the move from the left to the right so that your experiments can actually automatically move to the right, then it’s more like a reporting tool that gives you, at any point in time a good picture of where you are with your, optimization ideas. Now let’s assume that, David, we are here. We have found enough issues. We have a good prioritization workshop. We have a good concept for it. So now is the moment of truth, the 1st optimization sprint. Also, use VWO to build a test, we’ve got even a mock-up from our UX expert that’s so that we all align how the variant should look. So what is typically the first step then when you launch your optimization program?

David:

Right. So as you were saying, some of the ideas or hypotheses will require design or UX. So let’s assume that we’ve done that, or maybe the idea that we want to execute doesn’t need a design because the one that searches, as Sergio mentioned, just removes elements from the navigation. We don’t need the design. So what I would say is we need to define, and it’s based on the scoring, how complex an experiment is.

So if it’s easy to do and up to an extent, we will be doing it super easily with the visual website optimizer visual editor. Right? VWO’s visual editor allows a huge amount of additions, removing elements, adding elements, moving things around, replacing images, text, links, whatever. So for a lot of experiments, it is enough, but there are some that might require custom JavaScript code, and those will require a developer to actually call the thing. Right.

Jan:

Yeah. Maybe, but here, let’s build a test. I think that’s what everybody’s here and, show that it’s not either or. I think, some years back, someone at VWO made a very smart decision and said, you should always be able to switch between using a front-end editor with dragging and dropping. So, people like me, don’t know how to write a single line of code. And David and other experts who actually prefer to work, by coding. Let’s see, if we build a test, what is the first step normally? Sergio, before we go into building the modifications.

Sergio:

Yeah. Well, the first step would be the platform asking us where. Where are we going to build that test? Right? So we can select the URL where we are going to build that test.

We can all, of course, we can set 1 URL and make it simple, you know, I’m going to beat that. I think this webpage but we can also build it on more than one page. Right? Let’s think about maybe, newcomers. Right?

We want to build a test through all the product pages, and we want to gather the information through all the product pages. We can do that. Just by, clicking here on the URL, but we can also use some other, matching criteria, like making it much with some pattern or with some regular expression. And we can even exclude some pages, right, we can do that. But let’s imagine now we’re going to build it in a simple way.

So the first thing would be the where. The second important thing would be who is going to become, part of that test. Well, we have a lot of options, according to the different plans of the tool. Of course, we have first the standard segments, which we can, directly filter, and test for different people, according to different sources, different device types, or different visitor types, but we can also use the custom segments, right? In the custom segment, we have unlimited options to segment here, right, for parameters, UTM parameters, we have JavaScript variables, we have, cookie values, we have, integration with some third-party applications we can get, audiences directly from Google audiences through the Google tag manager data layer.

I mean, we have a lot of options and also something related to the behavior, which is interesting. With the number of pages someone visited or the time spent by someone, if we need a little bit more, we can always create a custom trigger.

David:

Right. I think that is the first moment when we see one thing, that the ultimate level of optimization kind of automatically drives you into personalization. Take the first network. The network I said before personalization is segmentation. Yeah. And I was thinking that now with, the sunsetting of Google Optimize, you guys are gonna have a better integration with GA and its audiences. Right? So that’s correct. It’s really important.

Jan:

Yeah. Everybody asked that question. So, yes, to put it out there. It didn’t, in a couple of weeks, I think Google will announce the deeper integration with VWO, it’s done. It’s gonna be there, and you will be able to use the segments that you already specify within GA4 will be a tremendous step forward.

I think that was one of the reasons it’s part of Google’s decision to sunset Optimize and to build a deeper integration with tools like ours, to enable this. I think this is good, let’s say, the question of where to run the experiments and then who should see the experiments are the first step but then the front-end editor plays a vital role. I mean, that’s why VWO stands for visual website optimizer. But, I think there’s more to it. And, maybe, secondly, we can show how we, someone like me without knowing how to write code can actually modify a website, in a few minutes.

Sergio:

Sure. That’s right. The visual editor from the experience I have of speaking with people. It definitely works better than Google Optimize, a visual editor, and some people who have used it in the trial version of VWO have said that they really see some improvements there the main thing, and it’s related to what David was saying, and Jan, VWO is a very flexible and very versatile platform. Right?

It can be used, of course, by Jan or by me who doesn’t know how to write code, but it can also be used by some developers. Right? So you can inject directly the JavaScript CSS or HTML code, see your variations, and create your variations from there. But you can also, you know, just click in some text and say, okay. We’re going to try a different headline here, and we’re going to say, awesomeness made the CRO platform.

Okay. We can actually write anything we want just modify any kind of test. That would be one thing we could do, right, modify the test, but we can also do other things like modify some styles, right, make some changes in colors or size, or anything. We can say, okay, we’re going to try this color with this button see if this triggers something in the visitor, right, and make them click more. That would be the second thing we could do.

The third thing we can do is we can hide things. Right? We were speaking, previously about hiding some elements. This is a really fast, way to test things, and then I definitely recommend it always. What else can we do?

We can move things, we can just click on one section and then say, ‘Hey, I want to find out how is this going to, impact the usage of the people, you know, the, the behavior of the people on the website. We’re going to place this here. And also, we can add some things. We can add some new things again without the help of the IT department. We can add some banners.

We have also some other pop-ups, such as subscription pop-ups. You know, you can modify anything. Here, if you want, you can change the URL. You’re submitting the information. You can set when this is going to be shown when someone is going to try to close the page or when someone scrolls a specific percentage of the page, you can also decide how frequently it’s going to be shown or when it’s going to be dismissed. I mean, the possibilities are actually almost unlimited.

Jan:

And I think it’s, so wrapping up what you just said. I can edit things both copies and styles and images so I can change things my favorite category with what you mentioned was hiding or removing things. I think there’s a whole bunch of really promising experiments, especially on mobile devices, to reduce things. I would say 8 out of 10 websites have too much showing too much stuff, especially on mobile when you only have a split second of attention. So hiding things delivers some really surprising results here.

And, moving things, copying, and pasting things. So there’s a whole bunch of things that you can do. But as we said earlier, there are people like David from another planet. Right? They know how to code, and, not like me.

And, so we can’t switch to the coding motor, right? So it’s okay. Maybe you can show just quickly if you were a programmer and say, oh, right. Here it is. At the bottom, there is a command bond way can actually go into JavaScript thing on CSS.

David:

Exactly. Really cool. Sorry, Sergio. I’m gonna interrupt you. From a developer’s standpoint, this will allow you to understand how VWO is interacting with your website and what it’s doing, and you can use your own code or you can adapt the code that the tool is providing.

So it’s really, really nice.

Jan:

Yeah. I think it’s, I mean, we should not confuse this with building a split test as well. We might be touching on this a little bit later. There are some tests with more complex and complete changes on the page where it could make sense that you rebuild a clone of your original page.

And then you host both pages and you just split the URL and some users are gonna see this page and the others, this page. This is a different story, which gives you the whole flexibility of coding whatever you want to code on your alternative page. I’ve seen this quite frequently, but here, it is really and I also see that when some of the UX designers have used the front-end editor and built some modifications, then sometimes the developer does some fine-tuning here to make sure that we actually have the right color fit, we have the right fonts, we have the right distances, and so on. So a lot of fine-tuning is happening in the code, I think. Right. Now we’ve got a visual, editor. We can switch it to coding mode. Let’s say I have another good idea, so how easy is it to convert an A/B test into an ABC test? 

Sergio:

Super easy. I mean, actually, you could turn it into the ABC until said or whatever letter you need. We have unlimited variations there. Just by clicking here, and adding the variation, we will be working on that variation. Right? I mean, you can build any variation that you want.

Jan:

I personally use a lot of cloning because once I have done all the modifications already, I have other ideas for the variant that I just created and then I clone this particular variant and, just add another detail to it, which gives me a perfect search variant. Yeah. So that’s done. That’s great. I think the widgets, we could speak about this forever.

It’s a fantastic tool. To really also work on your messaging, right, to create, to communicate. It’s much more than what you would just add. It’s also, that you can actually decide when a particular widget shows up or a message is being shown or a pop-up or a banner, how long it should stay, when it should be removed, and how often it should appear. So there’s a whole bunch of things.

I mean, I always say, okay, careful. Don’t overdo it. Then it’s also to be some expertise in it. We’ve seen websites where people are just doing this all over, and it’s full of pop-ups and stuff like that. So, you can hold a lot of wrong things with that, but thank god there are companies who take care that you don’t do this.

Well, then the modification is done. The test is running, let’s say. And then at some point in time, a week later or 2 or 3 weeks later, you get the good message from your platform, that you’ve got a winner, but a winner in terms of.. 

David:

Sorry, Jan. I’m gonna interrupt you because you missed a very important point here. QA. So we don’t need to, so everything is fine. You just edit it or create your variations with the visual editor or not, and then you’re ready to go. You need to QA. So that’s basic. You need to QA in different browsers and nowadays, everybody is Chrome, but Firefox and Safari exist, then you need to identify. And this is where data comes in handy what browsers are your users using? Yeah, using the browser LAN operating system. So Safari on iPhone, Safari on macOS, and you need to try and replicate all of those to ensure that what you’re trying to deliver is what they’re actually getting. And, also, since this is an experiment that is gonna be running on top of your side, make sure that you’re not breaking something else, or make sure that if you launch an experiment and then somebody from another department makes a change to the website that your experiment continues to work.

So that’s sort of really, really important. And I think in VWO, it’s called preview mode. You have the preview mode where you can actually review an experiment before launching, right, the more the name implies everything. But I would say that, in our experience, spending time in QA is worth it. So it’s something a step not to speak.

Jan:

Yeah. Absolutely. I think we’ve seen a lot of situations where it did not happen. I mean, sometimes it’s okay because you’re lucky. You didn’t miss something particular, but if you just change a copy, there’s not too much stuff I could break.

But if you change your model, you’ll move it back, and suddenly on a specific device, it just looks terrible.

David:

And just imagine. From all different colors and you’ve been there as well. You changed a copy, but maybe the website has 3 languages. And you change the copy and suddenly everything is shown in English.

And when you switch to Spanish or German, it’s still in English because you’re in QA. For example, Yep. And then the next step that I would say is also important before launching is, metrics measurement. How do we measure success? What are the goals of the experiment?

Jan:

Well, maybe you could show this, set it here. A quick look at this step that I absolutely miss. So we’ve done the modification. Now it’s time to find the metric, all the metrics that we want to track, and that should also be the primary way that should also be defining, who’s the winner. Right? How does that work?

Sergio:

Yeah. Exactly. I mean, in VWO, we have different ways to measure that. So, we definitely need to select the goal that is going to tell us that this was a change that impacted the business. In the goals we have for the website. Right?

So we could maybe create one metric for the clicks on an x button, for example, and, we would have a year to, say, clicks and where the selector path is, this one, for example. Right? And we can even..

Jan:

I’m sorry to interrupt you. I mean, could you do it for people like me? So the basic situation is, my conversion rate is my primary goal here, and my primary metric, visiting people, number of visits, or unique visitors, that come to my thank you for your purchase page. How would I build this?

Sergio:

Definitely. We would track in that case visit. Thank you page and where the URL matches the thank you page. 

Jan:

Okay. Alright. Oh, even I claim the URL contains thank you.

Sergio:

Exactly. Also that. Yeah. And we could also track clicks on buttons from the visual editor just by clicking on a button and say, track the clicks on this button. That’s really, really easy to do for the future.

Jan:

But I always have a primary metric, which decides whether or not we’re gonna have a winner here or not. Right?

Sergio:

Exactly. And, actually, we can create as many, goals as we need here. You know, we can create more than 1 and, set this, goal as the primary one or this one which is going to be the one that decides or declares a variation as the winning one.

David:

And I think that this is important sometimes when building the hypothesis, defining what is the goal, and making sure that we stick to that because, okay, we change some copy. And let’s say, let’s imagine the hypothesis. If we change the copy on the homepage, we will increase links to the product page, for example. Okay.

So we learned the experiment and clicks to the broad page are the same, but the bounce rate has reduced. That’s fine, but that was not our goal. So our hypothesis was wrong. Was wrong. Completely wrong.

Then we need to decide what was the effect. Whether we like it or not, the experiment was a failure because our hypothesis of if I change this, then this is gonna happen, then it’ll happen. Something else happened. Right? So that’s why having a primary goal, I think it’s important.

Jan:

Yeah. The funny thing is the other day, I met a couple in a very mature, company in Holland, doing a lot of conversion rate optimization testing activities. And they were actually testing a lot of stuff that they thought would not work. Right? So I am almost sure it will not work.

Might be a crazy idea, but I don’t believe that it will work. So I would say the vast majority of the stuff they tested was stuff where they thought this wouldn’t work. But sometimes the hypothesis is wrong. It actually worked. And, so it sometimes depends on what you mean, if you have this one experiment where you bat on in Europe, you put all your heart and soul and hope into it, and then you perceive it as a failure.

I think that’s totally wrong. I think you should really come to a speed of experimentation that you also test the many, many things of which you’re not really sure would actually work. But since it’s very easy to clamp it up very often, try it. Try it out.

David:

And fail fast. Actually, I would say that a successful optimization program is not how many winners you have, but how many learnings because they’re not losing, they’re learnings, and you need to keep learning all the time. You need to try 100 different things, and then you’re gonna find 3, 4, or 5, successful winners. Perfect. Yeah. That’s great.

Jan:

Yeah. We ran a survey a couple of months back and, with around, 100 companies or so, different maturity rates in terms of testing. And, the average in the Netherlands was around 140 tests a year which they were running. So that was the average. That’s, you know, twice as much as in Germany, I would say. So they’re really advanced and they understood that all progress in mankind was made by experiments. And, whether it’s in medicine or in space shuttles or online experiences. It’s all based on experiments.

Anyway, back to the subject. We’ve done this. We’ve defined a metric. We ran some really proper quality assurance. Well, maybe someone in the team. I think you have a specific department for that, you know, David?

David:

Yeah. And that’s why we’ve tried to keep it different because usually if the same person who is developing something has to curate everything is gonna work perfectly. We need QA people who try to do unimaginable things. So what is your name? Banana. 1, 2, 3, 4, 5. So yes. And also, as part of our optimization efforts, what we usually do is define, a QA plan. So when we define the hypothesis, it is how are we gonna make sure that this works before we launch?

So we define what kind of tests the QA is gonna perform to ensure that everything works as expected. Right? Sometimes it’s easy because we just change the CTA and sometimes it’s complex because we are impacting, a transactional step on the checkout finance, for example. But, yeah, super important.

Jan:

Okay. Let’s just cover the full process, let’s continue. We’ve built this task. We ran the QA. We hit the start button. Data comes in. And after a couple of weeks, we see that either our variant or the control group has won. So, what do we do? I mean, what does that look like? Maybe, Sergio, you can share what is it, the information that VWO shares at the end of the experiment when actually, the statistical significance has been achieved and, we have some interesting results.

David:

Usually, what happens when you launch an experiment, the premise is that first day or two, people start to freak out because numbers are wild. And I would suggest patience and caution is, let’s give it a couple of days. So the tool has had time to mix enough people and whatnot. Right? Yeah.

Jan:

Yeah. Yeah. Good point. Have a look at it, but don’t panic.

David:

Right. Exactly. At least for 2 days. And if after 2 days, it’s still not working, maybe consider it, but 2 days at least. Yeah.

Jan:

Exactly. Oh, let’s be positive. How does the result look in VWO? And, what do I see here?

Sergio:

Well, of course, you have here your important metric, which is the conversion rate. And, this is, the way the platform declares a winning variation, right, according to the main goal that we have set up there. And one interesting thing that happens here in the video platform is you have, the possibility to access the heat map of one specific variation. Right? So sometimes as David says, we get a lot of learnings, you know, from testing.

We have an incredible hypothesis and then we have a very not expected result. So, the interesting thing about VWO is that you can separate the people, and the visitors from the different, control and other variations and have access to the heat maps of only the visitor from one specific variation. And why is this useful? Because it helps you, again, understand where is the problem or the success of your test. You can gather very, very useful and valuable information from there.

Jan:

This is basic. I think the really interesting stuff is happening when you click on the report, then we’re gonna see many more things, and we can do other things to better understand the data that came in. Right?

Sergio:

Definitely. We can, for example, filter. Right? Let’s imagine we have a test that’s running for all the traffic from your website, but then you want to say, okay. But what is happening, between the desktop and mobile visitors?

Okay. You can definitely have some important information from there, or you have different traffic, doing different things from, different countries So you can, separate again, that information again, even if you run that test for all your traffic, you have separated information after this is what we call the post-test segmentation. Right? And all the results you’re seeing here always in the dashboard, are according to the different goals you have set up here on the right side of the window. So, yeah, well, I mean, we have, a lot of useful information. You can create some other personalization campaigns from the data you have here. You can run some new hypotheses and ideas and run some new tests. I mean, it’s very, very interesting and helpful.

Jan:

I think that was one of my learnings when we worked together, David, that never give up on a test because you don’t really have a strong winner or so. And I think diving into the data really, there is a lot of stuff that you can extract.

David:

Yeah. And something else that I wanted to mention is that, make sure that you have enough data. So wait at least a couple of weeks to have a little bit of seasonality. That’s important. And what you just said then. So sometimes you need to dig deeper because it’s not working for the vast majority of users, or it’s not going as you expect, but if you start digging into it, you can see that there are specific segments that are reacting more positively to your experiment. And lastly, sometimes, and a lot of times, it’s gonna happen that your experiment is doing nothing. No impact. Negative or positive, and it’s a flat line. Just move on. I trade and learn I would say that for us is very important – iterating and learning.

Jan:

Yeah. I think that covers more or less, let’s say, the formation of the test from the beginning to the end. I would like to give our audience the last chance here because I don’t have a lot of questions for me. And, for the next meaningful questions, I would like. Then the one in the audience, listen up people, wake up. The one in the audience, who now asks a really interesting, meaningful question here, is gonna be invited to a pizza.

I sponsor this personally. I can invite you to either in Palma, I pay for the pizza. You plan here and pay for your flight. Or I’m gonna send this to your home office, whatever you prefer. But come up with an interesting question, please.

We’ll be, really, a friendly gesture here. I can understand that you are really amazed by what you saw. Yeah, I think so the test has been done. And, we had the saying before after the test is before the test. David, tell us a bit about the iteration of tests. How and what now do they want?

Wouldn’t know what’s the purpose of it in there.

David:

I was just thinking that this is, like, the circle of life. We started with analyzing data, and we ended up analyzing data again from an existing test that was launched. And then we can come up with the hypothesis or ideas or insights, and then we can iterate. We can take an existing test and VWO makes it easy to start a new one from an existing one. And you can check. Maybe you want to change it, as Sergio was changing the CTA color.

Maybe you want to try a different color. Maybe you want to make the CTA bigger or smaller and add an icon. So that’s iterations. And if you have a winner or a loser, it’s interesting to run iterations. So don’t get discouraged.

Keep pushing. If you feel like that is what you need to change, try it. I mean, just go for it. As long as you’re learning something, that’s good.

Jan:

Great. So, I think, we have covered the most important steps in building a task for all those guys out there who are still now considering, what do we do. What’s our next platform? And what should we take into consideration when we choose the platform? Well, David and I have worked a lot with several ones. Of course, I’m now at VWO. And, you could say I’m kind of biased, but from the bottom of my heart, VWO was my favorite tool before I joined VWO. It was.

David:

I know that.

Jan:

Yeah. You know that. I was only fighting. Because I really liked it. Probably it’s because I don’t know why. I think one of the important factors was they had great support, and we still have.

David:

That’s true. So I had, for the past 4 or 5 years, I’ve had a presentation made comparing different tools because as you know, we work with, the main ones. I’m not gonna say all of them, but the main relevant ones on VWO were one of them. So we had, 1 or 2 slides with pros and cons, of course. And I would say that I mean, we are done making science. We’ve been using VWO. I personally have been using it since it started 13 years ago. I don’t know. So support is one of the agents’ strongest points.

You’re right. Ease of use. And easy to get people on board because optimization or conversion rate optimization is not a one-man job. You need to get, I mean, not the whole company, but at least the whole marketing department involved, and change the mentality. So having an easy-to-access tool with, a low learning curve, like VWO. VWO is complex. Don’t get me wrong, but the starting learning curve is quick. So people can get junk into it quickly. So I think that makes it an interesting tool to start optimizing. And it’s reliable. It’s fast. So I don’t know. It’s really good. And also has, as I said before, the technical side, if you are a techy person, you can get there and do your things, without the visual aid. So yeah.

Jan:

David, I think you also earned yourself a pizza. I hope the next time I’m going to Palma there. So this is all for you. Thank you for your friendly words. It was great to see you again. And, thanks again to Sergio, for helping us see these tiny little sneak previews of the platform. Of course, this didn’t show everybody the entire process how to use the tool, and so on, but the good news is we are here to help. So as I said, if you would like to really see, or have a quick walk through, the platform, Sabrina and I will be happy to meet you here. As I said, my email address is jan@vwo.com.

You can also contact me, of course, if you would like to get in touch with David who knows more than me about conversion rate optimization, and he is really, an expert in this particular field and is happy to help as well and to connect with you. Thank you very much, gentlemen. It was a pleasure to meet you here. See you soon again.

David:

Have a nice one. Thank you.

Sergio:

Bye bye.