BLOG

on Conversion Rate Optimization

AB Testing Bulk Discount Offers Increased Average Order Value by 18.94%

Paperstone is a London-based office supplies company. They sell over 20,000 products throughout the UK. And also manage in-house the development and optimization of their online store, www.paperstone.co.uk.

Paperstone

Test Background and Hypothesis

Almost everyone expects a bulk discount when ordering for office supplies. Paperstone got to know about this significant detail during their customer survey. And like every successful business, they took their customers’ words seriously.

The team was thus all set to launch the bulk discount deals on their website.

Often when businesses implement something that has been requested by customers, it turns out to be profitable. But just like AB testing the price, bulk discount deals are a little tricky. You might see an increase in the number of sales, but this need not necessarily mean an increase in your profit margin.

So, it is important that the number of items sold by you increase so dramatically that it covers up for the lost margins in giving discount on bulk purchases. And of course, it should provide additional profit too.

This was a crucial test for George Harris, the Director of Paperstone, and his team, as results could go either way. The hypothesis was that bulk discount deals on selected products should increase profit margin from the website.

This was the Control page for the test:

Paperstone Control

A site-wide test was implemented. Bulk offers were promoted on some of the popular products.

This is how the offer was displayed on the Variation page:

Paperstone First Test Variation

Surprisingly, the bulk discount offers didn’t make any difference to their conversions. The most obvious thing to expect when you give bulk discount deals is, increase in sales. If it turns out to be profitable or not is a different issue. Here, the offer was ignored by visitors completely.

It seemed that the new design created “banner blindness” for visitors. The positioning of the offer in the right sidebar only contributed more to this loophole. This was contrary to the team’s expectations who believed that the “bright yellow bulk-buy ribbon” would actually draw visitors’ attention.

After this initial failure, the team faced a lot of pressure to ditch the idea and move on to other AB tests. This was also quite a complicated test to run and track. After all, not all visitors who came to the site might be interested in buying products on which bulk discounts were available. So, the test should also ideally run longer to be absolutely confident about the results.

Despite these challenges, the team decided to give another shot to the same hypothesis. They used Visual Website Optimizer for revenue tracking of their test.

Given below is the new variation they used for their second test:

Paperstone Winning Variation

This new variation was tested against the same Control that is mentioned earlier in the post. Here is the comparison image that summarizes the two tests for you:

Paperstone Comparison Image

Results

Each version received more than 10,000 visitors. The new Variation beat the Control and increased average order value by 18.94% for Paperstone. The increase in average order value for bulk discount offers was 5%. The revenue uplift was 16.85%. It is worth noting here that even though the bulk buy offer was available on selected products only. Still, there was a huge increase in the total average order value.

Next time you have a failed test, instead of discarding the entire hypothesis, make sure you analyze the execution for any loopholes as well.

Comments (2)

Leave a Comment
  1. Did they not use average quantity as a KPI for the last test?

    Removing the underline and renaming the Favourites link could also reduce destraction from the Add to Basket button, which also appears longer in the new variation. Did they also change the width of the sidebar?

    The yellow banner also caused the delivery date box to move below the fold or so it seems. Maybe this could have a side-effect as well?

    Interesting case never the less!

  2. Hi Paul,

    Thanks for the comment – the sidebar width stayed the same although the yellow banner indeed pushed the delivery info further down the page which could be culpable.

    We saw a 15% increase in avg lines per order, so we definitely saw this improve. However, it’s relatively easy to increase average lines per order (and revenue) in the short term if you’re lowering prices. One of the major complexities is the longer term analysis of how customers who purchase multibuy products behave further down the line in terms of profitability / CLV.

Leave Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes : <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

Contact Us / Login

Product
Resources Home