BLOG

on Conversion Rate Optimization

A/B testing tricks: How to test additional (new) content in a split test?

In a typical A/B split test, you test whether replacing an image, headline or button with a variation increases conversion rate. Using Visual Website Optimizer, this is fairly simple. You select the element, create a variation and then VWO replaces the selected element with the variation. Simple and sweet.

However, you may also want to test additional content on a website. That is, test a new element that doesn’t exist on your (control) webpage. For example, if you want to test if adding a new widget to the sidebar decreases bounce rate, how are you going to do it? Remember that you cannot simply do a traditional replacement A/B test because widget doesn’t yet exist. You have two options for such a test: either add an empty HTML tag (such as <div></div>), then simply select that empty tag and create a variation. Or, you can create a completely new page with a widget and do a split URL test.

Unfortunately, both of these options require you to modify your website in some way. What if you don’t want to involve IT team? One of our users, Karsten Lund, faced exact same problem and we helped him do this test without touching the website code or creating any new pages (note that the website already has a common VWO tag that allows one to create any number of tests from VWO interface without touching the website code).

First, here is the control page looks like:

Karsten wanted to test if adding a navigation menu on the left would increase downloads and sales. He was also open to removing the right sidebar as the left one will serve that purpose. To do this A/B test, we used Visual Website Optimizer’s Advanced code mode and added a CSS/JS section in the test. Then we created a variation with following JQuery and CSS code:

<div id="vwo-new-section">
   HTML content for new section
</div>

<style>
   .catalog { visibility: hidden; }
</style>

<script type="text/javascript">
   vwo_$(document).ready(function(){
      vwo_$(".catalog-right").remove();
      vwo_$("#vwo-new-section").css("float","left").css("margin-right", "30px").prependTo(".catalog");
      vwo_$(".catalog").css("visibility", "visible");

    });
</script>

What the code does is this:

  • <div id=”vwo-new-section”> is the left menu that Karsten wants to test in the test
  • In the style tag, .catalog { visibility: hidden; } hides the container section in the site (so that when a new section is added, it isn’t done in front of visitor’s eyes)
  • vwo_$(document).ready means execute the code once page has been loaded (we cannot add the left menu before page is ready because container section is not yet loaded to add the menu)
  • vwo_$(“.catalog-right”).remove(); removes the right menu
  • vwo_$(“#vwo-new-section”).css(“float”,”left”).css(“margin-right”, “30px”).prependTo(“.catalog”);
    floats the new section to the left, adds a right margin and adds it to the container element (.catalog)
  • vwo_$(“.catalog”).css(“visibility”, “visible”); displays the container element (we had hidden it earlier)

Note that all the code above is inserted in the <head> section of the page, that’s why the container element remains hidden and we wait for document to load before adding the left menu and showing the container element again.

Here is how the variation with left menu looks like:

It works perfectly and we are quite happy that we could help Karsten in setting up this test. Benefit of taking this approach was that Karsten did not have to involve SlideShop’s developers for making changes into the website so the turnaround time for getting the test up and running was quick (less than an hour).

Hopefully you like the technique, if you have any other such tips and tricks for Visual Website Optimizer, do let us know.

Founder and CEO of Wingify.

Comment (1)

Leave a Comment
  1. Great post, thanks for sharing! I’ve got some similar navigation tests I can’t wait to try out and this has provided excellent inspiration.

Leave Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes : <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

Contact Us / Login

Product
Resources Home