One of the things that make A/B testing terribly exciting is its ability to throw up results that go against popular perceptions. Data can sometimes prove wrong all your instincts, gut, intuition, experience or whatever you rely on to make strategic decisions. This case study is the perfect example of such an A/B test.
BrookdaleLiving.com offers various community living solutions for senior citizens. It offers services to those planning on living independently to those needing caregivers or nursing help.
BrookdaleLiving’s ‘Find a Community’ page was incredibly underutilized with just a bare-bone structure. It was very stark with no graphics, testimonials or content to encourage the visitor to convert. This is how the page looked:
Brookdale hired digital marketing and analytics firm Fathom to optimize its ‘Find a Community’ page. Matt Fieldman, Senior Account Executive at Fathom, completely redesigned the page by adding content, testimonials, credibility logos, USPs, among other elements. He used Visual Website Optimizer to create two versions of the page.
The first variation had a photo of an elderly woman. This is how the photo version looked.
The second variation had a 1 minute 56 seconds heart-tugging video in which many elderly people talk about their positive experiences with Brookdale. In the second version, the video replaced the woman’s photo. Everything else on the page remained the same. Matt expected this version to perform better than Variation 1.
This is how the video version looked (And here’s the video. It will auto-play in a new tab)
The image version outperformed the video version. When pitted against the original, the image version recorded a 3.92% increase in ‘Find a Community’ searches while the video version recorded just a 0.85% increase in searches. The image version also achieved the statistical confidence of 99.99%. The test was run for 2 months on over 30,000 visitors.
According to Matt, the seemingly modest 3.92% increase in community searches will result in a more-than modest $106,000 additional monthly revenue.
Why the variations worked?
Matt used Fathom’s most-successful landing page template while redesigning the ‘Find a Community’ page. He made sure to speak to all the four major online personas by adding relevant elements for each of the types.
These are the four kinds of online personas Fathom targeted:
1) Methodical consumer
This user is detail oriented and tries to read up every bit of information present on the page. Matt used additional explanatory content to speak to this category.
2) Spontaneous consumer
This type is quick with the buying decision or already has the intent of purchase while going to a website. Matt used a clear and concise search box with an arrow pointing towards the ‘City’ field to speak to this customer type. He also cuts down on the number of fields — a practice that has increased conversions for many Visual Website Optimizer customers.
3) Humanitarian consumer
This type wants to know who you are and whether they can place their trust in you. Matt uses two tried and tested elements to ward off customers’ fears — testimonials and logos. WikiJob, a UK-based VWO client, tested a customer testimonial on its product page and saw 34% increase in conversions. Similarly, another company increased conversions by over 72% by adding a trust badge.
4) Competitive consumer
As the name suggests, this type wants to know why they should choose you over the competitors. Bullet points calling out Brookdale’s unique selling points (USPs) were put up on the page to get their attention. A VWO client increased CTA clickthroughs by almost 50% by changing its CTA text to a better value proposition.
Why the image version worked better than the video version?
It will be fair to say that Matt wasn’t the only one surprised by the results. Given an option to choose between a video or a photo, most people have no difficulty opting for the former. Videos are expected to miraculously make your product and website look super sexy. Here’s why videos are hailed:
- Zappos’ sales went up from 6% to 30% by using video demos on product pages.
- Stacks and Stacks found that those shoppers who saw videos on its product pages were 144% more likely to add to cart.
- CrazyEgg got an explainer video made which helped them generate additional income of $21,000 per month.
So it’s no wonder Matt was left quite surprised by the results. “It’s the counter-intuitive nature of the result that was most fascinating,” he said.
The possible reasons why the image version worked better than the video version are:
1) Brookdale is already an established brand and the video acts as a distraction
The first hypothesis is that because Brookdale has been around for over three decades and is also listed on Nasdaq, it’s already a big brand and people don’t need much convincing to convert on the website. Visitors have usually done their research and arrive on the website knowing what they want, but the marketing video was a distraction.
The video has a lot of elderly people talking about specific points such as ‘food’ and the ‘caring nature of the staff’ at the communities. There’s a possibility that the visitors did not have these criteria in mind and only after listening to them thought of ‘food’ and the ‘qualities of staff’ as important factors in deciding on a community. They possibly then went back to read up more on what the competitors are offering.
2) Most users fall into spontaneous customer type
I am not so sure of this hypothesis since I feel community living solutions are seldom spontaneous decisions, but I would love to be corrected if you think I am wrong. Matt himself feels that the customers want to convert quickly and are not interested in the video.
“As you can see, we were wrong. We now understand our customers better: they aren’t interested in marketing videos so much as they want to convert quickly and easily on the page,” he said.
3) Target audience has slower internet connection so videos might not work for them
I used Alexa to find out that Brookdale’s target audience is over-represented by women who didn’t go to college and who browse internet from home.
Now, a report on average home internet speed in the US tells us that people who didn’t go to college are less likely to have high-speed broadband access than those who went to college.
Now by adding the two statements, we can come to a hypothesis that majority of Brookdale’s audience is represented by women who don’t have access to high-speed broadband access. The slower internet speed might be leading to a painful video watching experience, and hence the image version worked better.
Note: The third hypothesis was added to the post following a very valuable point made by Tommy of ConversionXL in the comment section.
Do you have a theory?
Can you think up of any other possible explanation why the non-video version worked better than the video version? If so, please share them in the comments below.