Homestead, PA (US)
Smiley Cookies was started by Jim Broadhurst in 1986, as part of the family dining chain called Eat’n Park Restaurants, which was in existence since 1949. They allow you to gift customized cookies to anyone for any occasion (weddings, sports events, corporate events, etc.).
Jim led Eat’n Park to become one of the nation’s most popular full-service restaurant chains with more than 75 restaurants throughout Pennsylvania, Ohio, and West Virginia. As a practice, the restaurant provides a free Smiley Cookie with each child’s meal.
Trinity Insight wanted to test which value proposition works best for SmileyCookie visitors. They wanted to understand which aspect of SmileyCookie’s service is more important for visitors.
SmileyCookie has a large header location, which they use for seasonal or promotional offers. It is at a prominent place and hence catches visitor attentions. As an eCommerce store can position itself in various different ways, Trinity Insight wanted to test which value proposition works best for SmileyCookie visitors.
In other words, they wanted to understand which aspect about SmileyCookie’s service is more important for visitors. Following is a screenshot that shows the header where these value propositions were tested:
Trinity Insight tested the following 5 different value propositions and designed their respective creatives:
The following screenshot depicts the creatives that were developed:
While the test was running, the overall conversion rate for the site was 8.98% with an average order value of $29.85, yielding a $2.68/per visit value.
Over this same period of time, the winning creative, Order Today -> Ships Next Business Day, had a 12.61% conversion rate and a $29.95 average order value resulting in a $3.78/per visit value, a 41% increase over the site average. Note that this increase was statistically significant at 95% confidence level.
Cookies are perishable items; and when you order cookies as a gift or for a special occasion, you want them sooner than later.
Next day shipping increased purchases because it tells the visitors that they don’t have to wait for long to get the cookies. There is no uncertainty, and visitors know exactly when they are going to get the cookies.
As these cookies are ordered as a gift or for special occasions, customers may not be very price conscious; therefore, value proposition regarding discounts or low shipping rate didn’t work that well.
Of course, all this is obvious after we have A/B test results. In fact, all the reasons proposed above may not be true. Interpreting A/B test results is always difficult because if you know why a variation works better, you don’t need to do A/B testing in the first place. You can just implement that variation directly without A/B testing.
Nevertheless, interpreting A/B test results is useful to note what may work in the future for other similar websites. So, if you sell similar perishable items online (say cakes, food, ice-cream, and others), emphasizing how fast your shipping is may increase your purchases. Try A/B testing it!
Thank you for your time.