Commercial Services & Supplies
This case study is based on a case study originally written by George Harris, Director of eCommerce at Paperstone.co.uk.
Paperstone is a UK-based online office supplies company. The UK market has a couple of dominant players, but many smaller companies sell similar products. Price is the primary basis on which the smaller players compete for business. Paperstone uses VWO to optimize its website.
A challenger seeking to target the large, established brands in the UK office supplies market, Paperstone wanted to use its website to make buyers aware that it offered products at prices lower than what the larger players offered. Paperstone believed that doing so would increase sales and hence market share.
George and his team hypothesized that displaying their competitors’ higher prices on 5,000 product pages will increase clicks on ‘Add To Basket’ and their overall website conversion rate.
This is what control looked like:
The team decided to display competitor prices along with Paperstone’s prices on the relevant product pages of the web site. However, as the team considered this change, concerns arose about the possible impact on visitors/ buyers:
Finally, the team decided to set up the test so that visitors became a part of the test only if they visited one of the 5,000 product pages (out of 18,000) that had competitor price data.
This is what variation 1 looked like:
The test was run for over 12,000 unique visitors (50/50 split between control and variation). The test goals were to measure “add to basket” and conversions to sales. On both counts, the variation recorded a poorer performance than the control. However, the results were statistically inconclusive.
A second round of tests was run using another variation in which the following changes were made:
This is what variation 2 looked like:
Using VWO, Variation 2 was tested for over 12,100 unique visitors (50/50 split between control and variation).
The second test was conclusive for an increase of 10.67% in conversion rate (at a 95% confidence) but inconclusive for clicks on “add to basket”.
Thank you for your time.