Trinity Insight wanted to test which value proposition works best for SmileyCookie visitors. They wanted to understand which aspect of SmileyCookie’s service is more important for visitors.

SmileyCookie has a large header location, which they use for seasonal or promotional offers. It is at a prominent place and hence catches visitor attentions. As an eCommerce store can position itself in various different ways, Trinity Insight wanted to test which value proposition works best for SmileyCookie visitors.

In other words, they wanted to understand which aspect about SmileyCookie’s service is more important for visitors. Following is a screenshot that shows the header where these value propositions were tested:

Trinity Insight tested the following 5 different value propositions and designed their respective creatives:

• Order Today -> Ships Next Business Day
• Want to save $5 OFF your next purchase? SIGN UP NOW ->
• $6.99 Ground Shipping For Your Entire Order
• FREE SHIPPING on any order over $40
• Cookies Made Fresh & Hand Iced For You!

The following screenshot depicts the creatives that were developed:

While the test was running, the overall conversion rate for the site was 8.98% with an average order value of $29.85, yielding a $2.68/per visit value.

Over this same period of time, the winning creative, Order Today -> Ships Next Business Day, had a 12.61% conversion rate and a $29.95 average order value resulting in a $3.78/per visit value, a 41% increase over the site average. Note that this increase was statistically significant at 95% confidence level.

Key Learnings

Cookies are perishable items; and when you order cookies as a gift or for a special occasion, you want them sooner than later.

Next day shipping increased purchases because it tells the visitors that they don’t have to wait for long to get the cookies. There is no uncertainty, and visitors know exactly when they are going to get the cookies.

As these cookies are ordered as a gift or for special occasions, customers may not be very price conscious; therefore, value proposition regarding discounts or low shipping rate didn’t work that well.

Of course, all this is obvious after we have A/B test results. In fact, all the reasons proposed above may not be true. Interpreting A/B test results is always difficult because if you know why a variation works better, you don’t need to do A/B testing in the first place. You can just implement that variation directly without A/B testing.

Nevertheless, interpreting A/B test results is useful to note what may work in the future for other similar websites. So, if you sell similar perishable items online (say cakes, food, ice-cream, and others), emphasizing how fast your shipping is may increase your purchases. Try A/B testing it!