It is interesting how counter-intuitive A/B testing results can get. One of the best examples of that counter-intuitiveness is a recent A/B split test. This test was done by a Visual Website Optimizer customer Vendio, an Alibaba.com company that specializes in providing free e-Commerce stores to its merchants. They have special landing pages for the free store signup and the objective of this test was to increase signup conversion rate on one of those landing pages.
A/B test details
Their original landing page (control) combined marketing content and registration fields in an attempt to reduce the number of clicks for a successful registration. This is how it looked:
Note that it uses the so-called “best-practice” of embedding the signup form in the landing page itself. They had long been using the layout of the original page because as a best practice they presumed that reducing the number of clicks for a registration increased conversion rate. Although the page was performing relatively well, they wanted to make sure the included registration fields weren’t too aggressive or limiting in any manner.
So, the variation that they tested had somewhat similar imagery and content but the page didn’t include any registration fields and had slightly different styling. Clicking on ‘Signup Now’ button simply took the visitor to a page with signup form. In other words, Vendio added an extra step in their conversion funnel. Not a smart move, huh? This is how variation looked like:
A/B test results
Guess what? The page without the registration fields performed better – much better – to the tune of a 60% increase in conversions! Here is what Vendio had to say about the results:
Best practices are NOT always true! It’s still hard to believe, but the numbers don’t lie.
So, that’s the biggest lesson here and it is worth repeating: “Best practices are NOT always true”. If you do changes on your website or landing pages without A/B testing them, you are actually flying in the dark. Another lesson here is that it is it is worth testing radically different ideas – which, on the first glance, may appear not-so-smart (like removing signup form from the landing page!).
Testimonial about Visual Website Optimizer
After the successful test, Vendio gave us a great testimonial:
Visual Website Optimizer was extraordinarily valuable. It was far easier to use than other solutions (some of which we couldn’t even get to work properly after implementation). Without VWO we would still be missing out on all those registrations, and we are continuing to see great and other surprising results with VWO.