Success Story

Adding Benefits To A CTA Led To Increased CTR For Dutchento

190.31%

Click-through Rate

Industry

Software

Company Size

1-10

Location

Amsterdam (Netherlands)

Capabilities Used

A/B Testing

VWO and Dutchento

Your test variations usually affect many different conversion goals on your site such as free trial sign-ups, paid sign-ups, newsletter subscription, and others. A similar instance of this was witnessed by Dutchento, one of the VWO customers.

Dutchento is the official Dutch community for Magento CMS, ran by Guido Jansen, a Magento specialist.

Objective

The goal of this A/B test was to increase subscriptions for Dutchento’s newsletter and RSS subscription.

Solution

The call-to-action for subscription is located in a box on all pages of the site. See the control version of subscription box below and note that there is no explicit incentive for a visitor to subscribe:

Control

Guido tested a variation of this subscription box which included a title and some benefits in a bullet list. Here is his hypothesis for increasing subscriptions:

People don’t just subscribe to a newsletter or newsfeed for nothing, you should convince them it has added value above just visiting the website. So what I wanted to test is if adding convincing reasons to subscribe would increase the newsletter and newsfeed subscription rate. I measured the impact of the convincing reasons on clicks on both the newsletter and newsfeed links.

Here is how his variation (of subscription box) looks like:

Variation

Conclusion

As expected, Guido saw a significant improvement of 190.31% in the newsletter click rate. However, the newsfeed click rate decreased (–44.46%) which did surprise him (and us!).

He expected that the convincing reasons would affect both positively, but apparently, it had a negative effect on newsfeed clicks.

The reason why clicks on newsfeed decreased is not clear, but we believe that the benefits in the variation were so compelling that visitors chose to get the blog updates via email (where they will be sure to read them) rather than the RSS reader (where they may miss them). A great way to get more insight into this would be to randomize the position of newsfeed/newsbrief to eliminate the positional effect of those links.

Guido used VWO for A/B testing, and here is what he had to say:

VWO was very valuable [for testing]. It’s the easiest A/B and Multivariate testing tool I know. It’s great not to be dependent on the development department to create and run your tests.