A/B testing and conversion rate optimization (CRO) are not synonymous, but often confused.
A/B testing is exactly what it says—a test to verify different sets of variations on your website. Conversion rate optimization, however, is much more than just testing.
Conversion optimization is a scientific process that starts with analyzing your business’ leaks, making educated hypotheses to fix them, and then testing those hypotheses.
Conversion optimization is a process that needs to be repeated, but A/B testing is a technique. A formalized conversion optimization process can advance somewhat like this:
- Tracking metrics and identifying what parts of the conversion funnel need fixing
- Analyzing why visitors are doing what they are doing
- Creating and Planning your hypotheses for optimization
- Testing the hypotheses against the existing version of the website
- Learning from the tests and applying the learning to the subsequent tests
To further clear up the air around the two terms, we got in touch with the top in line conversion rate experts and picked their brains on the same. The experts tell us about their experiences with A/B testing and conversion optimization and why you should switch to the latter.
Quotes from Conversion Rate Experts
Chris Goward, Founder and CEO, WiderFunnel
Back in 2007, I could already see that a huge gap was developing among companies that are perfecting a process for conversion optimization and those that are following the easy advice of so many consultants.
Instead of selling top-of-mind advice, I focused WiderFunnel on refining the process of continuous optimization for leading brands. For each of our client engagements, we run a holistic CRO program that builds insights over time to continuously improve our understanding of their unique customer segments. The results speak for themselves.
Ad hoc A/B testing is a tragic use of your limited traffic when you realize how much growth and insights structured optimization program could be delivering. In an example that we published recently, a structured CRO program is exactly what this company needed to double its revenue two years in a row, over the ad hoc testing it was previously doing.
Brian Massey, Founder, Conversion Sciences
The most effective conversion optimization program seeps into the bones of your organization. Decisions that were once exclusively creative in nature gain a data component. Much of the guessing drains from your online marketing. We call this “rigorous creativity,” and it marries your best marketing work with insights about your visitors. It cannot be accomplished by running a few tests, but comes from asking daily, “Do we have some data to help guide us? If not, can we collect it?” The rigorously creative business is good at finding and creating this data and using it to maximize visitor satisfaction and business profit.
Rand Fishkin, Founder and CEO, Moz
Without a strong CRO strategy that encompasses the experience visitors have discovering, using, exploring, and hopefully eventually converting on your site, you’ll always be plugging holes in a leaky bucket rather than building a better container.
The best opportunities to improve conversion usually aren’t from changing individual pages one at a time with a multitude of tests, but rather by crafting a holistic, thoughtful experience that runs throughout the site, then iterating on elements consistently with an eye to learning, and applying knowledge from each test to the site as a whole.
Karl Gilis, Co-founder, AGConsult
An AB test should come at the end of your homework. If you’re just AB testing, you’re probably gambling. Your tests are based on things you’ve read on the Internet, gut feeling, and opinions. Some of your tests will be winners, most of them losers. Because you’re shooting blanks.
The homework is data analysis and user research. This will reveal the problem areas and why your visitors are leaving or not doing what you want them to do. The better you know the dreams, the hopes, the fears, the barriers, and uncertainties of your users, the better you’ll be able to work out a test that will have a real impact.
In case you’re in doubt, impact seldom comes from design changes. Don’t change the color of your button, change the text on that button. Not randomly, but based on what users want and your knowledge of influencing people.
Don’t focus too much on the design. Focus on your offer, your value proposition, and how you sell your stuff.
Don’t sell the way you like to sell. Sell the way your customers want to buy.
André Scholten, SEO and Site Speed specialist, Google Analytics
Create a strategy that makes your clients happier and don’t focus on the money. Single non-related tests on the conversion funnel follow each other up, based on abandonment rates, judged on their influence on revenue. That’s not a strategy but more an operational process where test after test is conducted without vision. You should create a test culture within your company that tests everything that will make your website a nicer place for your customers. Give them feedback possibilities with feedback or chat tools to learn from these. Take their wishes into account and create tests to verify if their wishes are met. Create a test strategy that focuses on all goals: not only the money, but also information-type goals, contact-goals, etc. It will give you so much to do and to improve. That’s a holistic approach to testing.
Kathryn Aragon, Content Strategist & Consultant
“Winging it” may work for musicians and cooks; but in marketing, any decision made outside of a holistic CRO program is a bad one. Only through testing will you find the right message, the right audience, and the right offer. And only after you nail these critical elements will you see the profits you need. It doesn’t matter how small or new your business is, take time to test your ideas. You’ll be glad you did.
Joel Harvey, COO & Conversion Optimization Expert, Conversion Sciences
To say an online business is great due to AB Testing is like saying a Football team is great because of their stadium. It is the entire team framework that leads to winning. An optimization framework integrates A/B testing as one component that includes the team, the brand, advertising, and a solid testing strategy. This is how industry-leading websites win year after year.
Rich Page, Conversion Rate Optimization and Web Analytics Expert
Many online businesses make the mistake of thinking that A/B testing is the same as CRO and don’t pay enough attention to the other key aspects of CRO. This usually gives them disappointing results on their conversion rates and online revenue. Web analytics, website usability, visitor feedback, and persuasion techniques are the other key CRO elements that you need to frequently use to gain greatest results.
Gaining an in-depth visitor feedback is a particularly essential part of CRO. This helps you discover your visitor’s main needs and common challenges, and forms high-impact ideas for your A/B tests (rather than just guessing or listening to your HiPPOs). Gaining visitor insights from usability tests and watching recordings of them using your website is particularly revealing.
Peter Sandeen, Value Proposition and Marketing Message Development Expert
Just about every statistic on A/B test results says that most tests don’t create positive results (or any results at all). That’s partly because of the inherent uncertainties of testing. But a big part is the usual lack of a real plan.
Actually, you need two plans.
The first plan, the big picture one, is there to keep you focused on testing the right parts of your marketing. It tells if you should spend most of your energy on testing landing pages, prices, or perhaps webinar content.
The second plan is there to make sure you’re creating impactful differences in your tests. So instead of testing two headlines that mean essentially the same thing (e.g. “Get good at golf fast” and “Improve your golf skills quickly”), you test things that are likely to create a different conversion rate (e.g. “3-hour practice recommended by golf pros”). And when you see increased or decreased conversion rates, you create the next test based on those results.
With good plans, you can get positive results from 50–75% of your tests.
Roger Dooley, Author of Brainfluence
Simple A/B testing often leads to a focus on individual elements of a landing page or campaign – a graphic, a headline, or a call to action. This can produce positive results, but often distracts one from looking at the bigger picture. My emphasis is on using behavior science to improve marketing, and that approach works best when applied to multiple elements of the customer journey.
Jeffrey Eisenberg, CEO, Buyer Legends
Conversion rate (CR) is a measure of your ability to persuade visitors to take action the way you want them to. It’s a reflection of your effectiveness and customer satisfaction. For you to achieve your goals, visitors must first achieve theirs. Conversion rate, as a metric, is a single output. CR is a result of the many inputs that make up a customer experience. That experience has the chance to annoy, satisfy, or delight them. We need to optimize the inputs. Ad hoc A/B tests cannot do this. Companies that provide a superior experience are rewarded with higher conversion rates. Focus on improving customer experience, and you’ll find the results in your P&L, Balance Sheet, and Cash Flow statements.
Jakub Linowski, Founder & Lead Designer, Linowski Interaction Design
Thinking beyond the individual A/B test as optimization is a natural part of gaining experience. We all probably started off by running a handful of ad hoc tests and that’s okay—that’s how we learn. However, as we grow, three things may happen which bring us closer towards becoming more strategic:
1. We become conscious of ways in which we can prioritize our testing ideas.
2. We become conscious of the structure of experiments and how tests can be designed.
3. We think of a series of upcoming tests which may or may not work together to maximize returns.
Here is one example of one test strategy/structure: The Best Shot Test. It aims to maximize the effect size and minimize the testing duration, while doing so at the cost of a blurred cause-effect relationship.
Naomi Niles, Owner, ShiftFWD
Running basic A/B tests based on best practices is okay for a start. But to really get to the next level, it’s important to see how all the pieces of the puzzle fit together. This gives us a better understanding of what exactly we’re testing for and reach for results that fit the specific goals of the organization.
Kristi Hines, Certified Digital Marketer
Depending on your business and the size of your marketing team, you may want to go beyond just testing your website or a landing page. You may want to expand your A/B testing to your entire online presence.
For example, try changing your main thing (keyword phrase, catch phrase, elevator pitch, headline, etc.) not just on your website, but also on all your homepage’s meta description, your social media bios and intros, your email signatures, etc.
Why? Because here’s what’s going to happen. If you have consistent messaging across a bunch of channels that someone follows you on, and all of a sudden, they come to your landing page with an inconsistent message (the variant, if you will), then they may not convert simply because of the inconsistency of your message. Not because it wasn’t a good message, but because it wasn’t the message they were used to receiving from you.
As my own personal case example, when I change my main phrase “Kristi Hines is a freelance writer, business blogger, and certified digital marketer.” I don’t do it just on my website. I do it everywhere. And I don’t do it for just a week. I do it for at least two to three months unless it’s a complete dud (i.e., no leads in the first week at all).
But what I usually find is when I find a good phrase, I’ll start getting leads from all over the place. And usually they will say they went from one channel to the next. Hence, don’t just test. Test consistency across your entire presence, if possible. The results may be astonishing.
Jason Acidre, Co-founder/CEO, Xight Interactive
I do think that Conversion Rate Optimization as a marketing discipline goes beyond just a series of A/B and/or Multivariate tests. As external factors such as your brand and what other people say about the business (reviews and referrals) can also heavily impact how a site can perform in terms of attracting more actions from its intended users/visitors.
For instance, positive social proof (number of people sharing/liking a particular product or a brand on different social networks) can also influence your customer’s buying process. And improving on this aspect of the brand involves a whole different campaign – which would involve a more holistic approach to be integrated to your CRO program. Another factor to consider is the quality of traffic your campaign is getting (through SEO, PPC, paid social campaigns, content marketing, etc.) The more targeted traffic you’re able to acquire, the better your conversions will be.
A full-fledged conversion optimization program goes a long way and is a lot more beneficial than ad hoc testing.
So what are you waiting for? Let stepping up to conversion optimization be your #1 goal in the new year.