Dutchento Tested a CTA That Communicated Benefits and Increased CTR

The goal of this A/B test was to increase subscriptions for Dutchento’s newsletter and RSS subscription.

The call-to-action for subscription is located in a box on all pages of the site. See the control version of subscription box below and note that there is no explicit incentive for a visitor to subscribe:

Guido tested a variation of this subscription box which included a title and some benefits in a bullet list. Here is his hypothesis for increasing subscriptions:

People don’t just subscribe to a newsletter or newsfeed for nothing, you should convince them it has added value above just visiting the website. So what I wanted to test is if adding convincing reasons to subscribe would increase the newsletter and newsfeed subscription rate. I measured the impact of the convincing reasons on clicks on both the newsletter and newsfeed links.

Here is how his variation (of subscription box) looks like:

As expected, Guido saw a significant improvement of 190.31% in the newsletter click rate. However, the newsfeed click rate decreased (–44.46%) which did surprise him (and us!).

He expected that the convincing reasons would affect both positively, but apparently, it had a negative effect on newsfeed clicks.

The reason why clicks on newsfeed decreased is not clear, but we believe that the benefits in the variation were so compelling that visitors chose to get the blog updates via email (where they will be sure to read them) rather than the RSS reader (where they may miss them). A great way to get more insight into this would be to randomize the position of newsfeed/newsbrief to eliminate the positional effect of those links.

Guido used VWO for A/B testing, and here is what he had to say: