The most exciting A/B tests are the ones which bust myths, defy general best practices and make the jaw drop.
The following case study is in the league of such intuition-defying A/B tests.
The Company
TheSolutionForDiabetes.com, as the name suggests, is a website that offers natural remedies for reversing diabetes.
Among the various solutions it offers is a free downloadable e-book on ways to control Type 2 diabetes.
The Goal
The website had a landing page for the e-book with an opt-in box asking for visitors’ e-mail address. Here’s how the landing page originally looked.
The Test
Martin Malmberg of TheSolutionForDiabetes.com wanted to see if adding a line about respecting the privacy of the visitors would affect the sign-ups. So he used Visual Website Optimizer to set up a simple A/B test. He created a variation which had the text — “We respect your privacy” — just below the call to action (CTA) button.
A/B Test Result
Martin’s hunch was correct. Adding the text about privacy did affect the conversions — but surprisingly negatively! The version with the privacy text registered a 24.41% drop in conversions! The test was run for over a month with the original recording a 99% chance to beat the variation.
What Really Happened Here?
You might be wondering what really happened here. Using any kind of trust indicator or copy that addresses visitors’ fears is usually considered a best practice. And why not? According to a survey, 36% online customers consider ‘identity fraud’ as their most prominent worry.
Then why did adding “We respect your privacy” right below the call to action (CTA) button decrease conversions in this case?
Possible Reasons Why the Variation Failed
1) The privacy text introduced fear
When asked why he thought the variation failed, Martin said, “I guess people started to worry about spam and privacy concerns after reading such a text. Whereas, the thought doesn’t even enter their heads without that text. So, at least in my niche, with my traffic sources, it seems best not to use such a text, even when worded positively.”
So instead of assuaging visitors’ apprehensions, the text had a counteractive effect as it instilled the seeds of distrust and worry in the minds of visitors.
ContentVerve ran similar privacy policy experiments on a sign-up form last year. In one of the experiments, it was found that the version with a privacy policy saying “100% privacy – We will never spam you” decreased sign-ups by 18% when pitted against a version with no privacy text.
2) The text acts as a distraction
This is just an extension of the first point. The privacy text right below the CTA might have been distracting visitors and took the focus away from the offer. It’s usually considered a good practice to keep the landing page as focused on the offer as possible.
Your views
Why do you think the version with the privacy policy failed? We would love to hear your views.
Comments (11)
I think there is another psychological trigger present here, and on an page that mentions privacy. It takes the user’s mind off their purpose of diabetes solutions and reminds them about their email inbox, where most people feel like they get too many emails already. And one more list sign up, no matter how private or trustworthy, will only add the daily stress of inbox overload and they must then decide whether the offer is worth it.
So yes, it’s a distraction away from the main CTA/goal and in an instant it gives them pause and hesitation and probably triggers guilt of all the unread emails. We know choices can cause indecision. When you remove the mention of privacy it keeps the user focused on the solutions they seek and they compete the simple form.
Another factor could be that “We respect your privacy” is so vague and unqualified (i.e., it doesn’t continue, “…and will never share your email address”) that it raises red flags.
It undermines the verisimilitude of the page’s other claims (which are kind of dodgy to begin with).
Hi there Mohita,
I found your article on Twitter and was a little disappointed to find only one split that led to the negative results. The author of the test should have tried to undo that fear and see if he could improve the results.
I’ve written about a similar test here a while back that shows how important the wording is and that a privacy policy or statement can increase conversions 🙂
http://www.iubenda.com/blog/2013/11/22/accept-privacy-policy-conversion-improvements/
It would be interesting to see how a few differently phrased privacy policies would perform. But I think you’re right that the privacy text introduced fear. Great point!
Excellent post and insights, Mohita.
Darren
@Rob
Completely agree with on how the guilt of unread mails might have acted as a deterrent. Thanks!
@Jon
Agreed. They could have worded the privacy policy a little differently. Adding some kind of trust sign might have helped as well.
@Darren
Glad you found the post useful. Do you want to suggest any variations of the privacy text?
@Simon
You make an interesting point. I have asked Martin to jump in the conversation. May be you could suggest some more test ideas around the privacy policy.
Well maybe a pointer to you following the legal obligations, you are protecting the user’s privacy according to the procedures outlined in the policy.
I feel like the test copy just sounds so amateurish that it will change people’s minds to the worse.
That Privacy Policy sentence do affect conversion.
(Had the same experiences with it)
And i agree with Mr Martin Malmberg’s conclusions on the Test.
Try to put yourself in the reader’s position, doesn’t the Privacy Policy scares you?
Doesn’t it makes you wonder whether can you trust the page owner with the privacy?
My current practice is to put the privacy policy link/statement somewhere else, not in the form area.
A test succeeded, a copy like:
We guarantee 100 privacy. Your information will not be shared.
The author just tell us the “failed” copy, but I do hope he could provide us more with another different copy words.