VWO Logo
Like this post?
Read our in-depth guide to A/B (or Split) Testing
Share this post
3 Min Read

Optimization v/s Validation – two distinct uses of A/B testing

Paras Chopra
Founder and Chairman of Wingify.

It is a myth that A/B testing can only be used for optimization and you need to fully establish a process optimizing it. In other words, many people think that A/B testing is best meant for big guys who already have an established conversion funnel, and now want to squeeze an extra 0.01% out of it. As a natural consequence to this logic, it may appear that A/B testing cannot be used at initial stages of a project (or startup) because there is nothing to optimize yet.

Wrong. There are actually two distinct uses of A/B testing: a) optimization; b) validation.

How to use A/B testing for Optimization?

As mentioned in the opening paragraph, this is the activity with which A/B testing is usually associated. Typical scenario: you are running an advertising campaign where you collect leads. You get 10 leads for every 100 visits and you realize at this conversion rate (10%), this activity isn’t profitable. So, you set out to optimize (or increase) the conversion rate. You test headline, product image, form length and a bazzilion different page elements. In the end, you manage to increase the conversion rate to 15%. Well done!

As you can see, with this A/B test, the objective was clear: optimize conversion rate for the activity one is currently doing. There are several VWO case studies where the tool was used for optimization. Examples include increasing sales by 20% by testing page design and increasing sales by 6.3% by testing Buy Now button.

How to use A/B testing for Validation?

Unlike above where we answer what is the best way to do an activity, in this case A/B testing is used to answer which activity to do in the first place. Validation means coming up with different possibilities and testing which one works best. Typical scenario: you decide to publish a new whitepaper for gathering leads. However, you are unsure which whitepaper will be most interesting to your customers. To answer that question, you make 5 different landing pages for each whitepaper and setup an A/B test. After the test is over, you check which whitepaper garnered most interest (via pre-release signups or some other conversion metric). Notice that you aren’t optimizing the lead collection process here, rather you are testing which whitepaper will potentially collect most leads.

To give you some examples, there are several VWO case studies where validation was carried out. Here is how a startup validated its positioning and here is how the appropriate position for a promotional message was tested.

Key Lesson

A/B testing isn’t just for big corporations who want to optimize the last drop out of their conversions. In fact, A/B testing is more useful at answering business questions. Want to know which product to launch next? Which feature to develop? Which new market to enter? A/B test it today!

What is your opinion about this? Did you ever use A/B test for validation, instead of optimization?

More from VWO

Archive A/B test results by automatically generating screenshots of variations

A multivariate or A/B test doesn’t end when you stop it. It is important to…

Read More
Paras Chopra

Paras Chopra

2 Min Read

Multivariate testing case study and tutorial on increasing conversion rate

I just published a guest post on Smashing Magazine titled Multivariate Testing in Action: Five…

Read More
Paras Chopra

Paras Chopra

Less Than a Min Read

Appsumo Reveals its A/B Testing Secret: Only 1 Out of 8 Tests Produce Results

This is the 2nd article in the series of interviews and guest posts we are…

Read More
Paras Chopra

Paras Chopra

5 Min Read
Shanaz from VWO

Hi, I am Shanaz from the VWO Research Desk.

Join our community of 10,000+ Marketing, Product & UX Folks today & never miss the latest from the world of experience optimization.

A value for this field is required.

Thank you!

Check your inbox for the confirmation mail