VWO Logo
Like this post?
Read our in-depth guide to Conversion Rate Optimization
Share this post
4 Min Read

8 A/B split testing case studies you cannot afford to miss

placeholder
Paras Chopra
Founder and Chairman of Wingify.

This post is a compilation of best A/B split testing case studies available on the Internet. In a way, this compilation proves that: a) A/B split testing really works; b) it doesn’t take much effort (hello VWO); c) you should be doing it right now! I think case studies are an interesting way to learn about a new domain. So, consider this post as a starting point for the wonderful world of A/B split testing.  Let’s start the list of case studies which you should definitely read today.

Writing Decisions: Headline tests on the Highrise signup page

37Signals did an A/B test on the headline of their pricing page. What they found was that “30 day Free Trial on All Accounts” had 30% more signups than the original “Start a Highrise Account”.:

Lesson: in a headline, the word “free” coupled with specific benefits (“30 day trial” v/s “start an account”) can do wonders.

Dustin Curtis’ “You should follow me on Twitter here”

The much hyped split testing case which involved testing of multiple different versions of call to text for twitter followers. Dustin found out that “You should follow me on Twitter here” worked 173% better than his control text “I’m on Twitter”.

Lesson: persuasive call to action “you should follow” proves to be much better than a passive call to action “i’m on twitter” (well, why would a visitor care).

Daily Burn, 20%+ Improvement

A simpler variation which had less options for a visitor to choose from resulted in 20% increase in conversions. The winning version was also much easier on eyes (in terms of details and text) as compared to the control.

Lesson: too many options on a page usually overwhelms the visitor and s/he simply bounces off the page.

A/B Test Case Study: Single Page vs. Multi-Step Checkout

Reducing a two page checkout process to one page checkout increased conversions by 20%. Though overall increase in sales and orders didn’t increase much, checkout % did increase.

Lesson: lesser the steps you have in your funnel, higher will be your conversion rate. A lot of visitors drop off at each stage of conversion funnel so try to shorten it as much as possible (ideally to one page).

How To Increase Your Adsense Earnings 94% Overnight

Sensationalist headline but interesting results. hey tried different combinations – ads in left v/s right; ads top v/s bottom; test ads v/s image ads. Not sure if the results are statistically significant, but definitely a good case study.

Lesson: proves that A/B testing not just works for product/sales websites, but can work for advertisement supported pages too.

Marketing Experiments Response Capture case study – triple digit increase in conversions

They demonstrated a 258% in opt-in rate. Focus was to remove all distractions and require the visitor to only provide email address. For completing his/her complete profile, the landing page incited the visitors with an Amazon gift card (which was again split tested).

Lesson: require the visitor to input minimal amount of information on the landing page, you can always ask for more information later (it related is the concept of sunk cost).

How we increased the conversion rate of Voices.com by over 400%

In addition to A/B split testing, they used a lot of other tools/techniques – clickmaps/user feedback/usability testing. The final result was an increase in 400% increase in conversions.

Lesson: simplicity; social-proof; role of video in conversions; clarity in design

Skype Boosts Homepage Conversion Rate with Radical Simplicity

Skype tested their homepage for 3 different variations. Key difference between the three versions was the level of details presented on the page. Guess which variation won the test? The simplest variation of all (see image above) saw an increase of 5% in downloads. The figure may not look like much of an improvement but when you consider that Skype may be getting tens of thousands of downloads each day, this increase becomes quite large and totally worth-it spending thousands of dollars on doing the A/B test. (But, hey, you don’t have too – after all, VWO is free to signup).

Lesson: simplicity, simplicity and some more simplicity

Do you know of any other interesting A/B split testing case studies which I should add in the post? Perhaps you may want to take help of A/B ideafox (a search engine for A/B split testing case studies).

More from VWO

Conversion Rate Audit: The First Step Of The CRO Process

Like everything else, there is a perfect timing to start with your CRO process. Rush…

Read More
Ruchika Sharma

Ruchika Sharma

9 Min Read

One-Stop Shop for eCommerce Conversion Optimization Resources

The eCommerce industry is incredibly competitive. While new online stores find it difficult to lure…

Read More
Nitin Deshdeep

Nitin Deshdeep

5 Min Read

Difference between A/B testing and multivariate testing

Many people seem to get confused about the terms A/B testing, split testing and multivariate…

Read More
Paras Chopra

Paras Chopra

2 Min Read
>
Shanaz from VWO

Hi, I am Shanaz from the VWO Research Desk.

Join our community of 10,000+ Marketing, Product & UX Folks today & never miss the latest from the world of experience optimization.

A value for this field is required.

Thank you!

Check your inbox for the confirmation mail