VWO Logo
Like this post?
Read our in-depth guide to A/B Testing
Share this post
3 Min Read

Optimization v/s Validation – two distinct uses of A/B testing

placeholder
Paras Chopra
Founder and Chairman of Wingify.

It is a myth that A/B testing can only be used for optimization and you need to fully establish a process optimizing it. In other words, many people think that A/B testing is best meant for big guys who already have an established conversion funnel, and now want to squeeze an extra 0.01% out of it. As a natural consequence to this logic, it may appear that A/B testing cannot be used at initial stages of a project (or startup) because there is nothing to optimize yet.

Wrong. There are actually two distinct uses of A/B testing: a) optimization; b) validation.

How to use A/B testing for Optimization?

As mentioned in the opening paragraph, this is the activity with which A/B testing is usually associated. Typical scenario: you are running an advertising campaign where you collect leads. You get 10 leads for every 100 visits and you realize at this conversion rate (10%), this activity isn’t profitable. So, you set out to optimize (or increase) the conversion rate. You test headline, product image, form length and a bazzilion different page elements. In the end, you manage to increase the conversion rate to 15%. Well done!

As you can see, with this A/B test, the objective was clear: optimize conversion rate for the activity one is currently doing. There are several VWO case studies where the tool was used for optimization. Examples include increasing sales by 20% by testing page design and increasing sales by 6.3% by testing Buy Now button.

How to use A/B testing for Validation?

Unlike above where we answer what is the best way to do an activity, in this case A/B testing is used to answer which activity to do in the first place. Validation means coming up with different possibilities and testing which one works best. Typical scenario: you decide to publish a new whitepaper for gathering leads. However, you are unsure which whitepaper will be most interesting to your customers. To answer that question, you make 5 different landing pages for each whitepaper and setup an A/B test. After the test is over, you check which whitepaper garnered most interest (via pre-release signups or some other conversion metric). Notice that you aren’t optimizing the lead collection process here, rather you are testing which whitepaper will potentially collect most leads.

To give you some examples, there are several VWO case studies where validation was carried out. Here is how a startup validated its positioning and here is how the appropriate position for a promotional message was tested.

Key Lesson

A/B testing isn’t just for big corporations who want to optimize the last drop out of their conversions. In fact, A/B testing is more useful at answering business questions. Want to know which product to launch next? Which feature to develop? Which new market to enter? A/B test it today!

What is your opinion about this? Did you ever use A/B test for validation, instead of optimization?

More from VWO

How to track conversions on subdomains or separate domains (3rd party apps)

If you want to track conversions for an A/B test (created using VWO) that aren’t…

Read More
Paras Chopra

Paras Chopra

2 Min Read

4 Ways To Increase Sales From Existing Users Using Web Push Notifications

Don’t we, marketers, have those days when we are stuck in traffic and wish that…

Read More
Anand Kansal

Anand Kansal

5 Min Read

How to Find Hidden Winners in Your A/B Tests

UC Berkeley was once sued for bias against women applying to their graduate programs. Their…

Read More
Samrat Krishna

Samrat Krishna

4 Min Read
>
Shanaz from VWO

Hi, I am Shanaz from the VWO Research Desk.

Join our community of 10,000+ Marketing, Product & UX Folks today & never miss the latest from the world of experience optimization.

A value for this field is required.

Thank you!

Check your inbox for the confirmation mail