Compare Courses Tested CTA Changes To Receive Unexpected Results

The Compare Courses website had 3 CTAs that generated leads:

• Send Enquiry form
• Ask an Expert column
• Request Brochure button

The main focus was to improve the overall conversion rate of the website.

Given the popularity of courses leading to a Bachelors of Accounting degree, the company decided to optimize the landing page for this course, which originally looked like this:

All 3 CTAs aimed at lead generation are clearly seen in the above image. While all 3 CTAs were equally important, the Compare Courses team wanted to test its hypothesis that conversion rate would increase by making changes to the lead enquiry form. They decided to redesign the Send Enquiry form.

This was the variation page they created:

As the variation page required significant code changes, Compare Courses ran a Split URL test that tracked all 3 lead generation routes for the variation page. The test was run using the VWO platform.

To make the page look better, the new Send Enquiry form was placed on the right so as to improve symmetry. Its placement was above the fold, and a testimonial was displayed next to it to provide social proof and enhance credibility.

The placement of the Ask an Expert form was also shifted to support the design symmetry of the page. It now appeared where the Send enquiry form appeared on the Control Page. The placement and design of the Request a Brochure CTA button remained unchanged in both versions.

In the world of A/B testing, optimization teams propose, but it is visitors who dispose. The results of the above a/b test were quite unexpected:

The Lead generation rate from Ask an Expert CTA on the page increased by 159.52% with statistical confidence of 99.99%.

The Request Brochure CTA recorded a 13.3% improvement in the lead generation rate with 99.99% chance to beat the original.

While the above results were positive, it must be remembered that no changes were made to these 2 CTAs except for the new placement of the Ask an Expert form to maintain design aesthetics of the web page.

On the other hand, considering the changes made to the Send Enquiry form, its placement above the fold and the addition of social proof, it was expected that lead generation would increase.

However, the tests showed that the variation performed much worse than the original, recording a 53.87% decrease in the lead generation rate. The original version of the page converted leads at 9.71% while the variation converted at 9.52%. Given the large number of visitors to the page, this drop meant a loss of 100+ leads from the variation.

Here’s the comparison image of the original version and the variation that were tested and the changes made:

The results of this test make it clear that when it comes to optimizing website conversions, one must never take anything at face value or go by what “feels” or “sounds” right. Even the most widely accepted “best practices” can belie expectations when applied without careful thought (and testing) to a website. Therefore, never make any changes permanent without having first tested them.

Another interesting takeaway from this test was that when there are more than one CTA on a page, it’s always best to track all of these during the test, even if changes have not been made to all of these.