{"id":49594,"date":"2020-02-10T17:26:24","date_gmt":"2020-02-10T11:56:24","guid":{"rendered":"https:\/\/vwo.com\/blog\/?p=49594"},"modified":"2025-05-05T16:40:25","modified_gmt":"2025-05-05T11:10:25","slug":"ab-testing-hypothesis","status":"publish","type":"post","link":"https:\/\/vwo.com\/blog\/ab-testing-hypothesis\/","title":{"rendered":"How to Create a Strong A\/B Testing Hypothesis?"},"content":{"rendered":"\n

Experimenting without a hypothesis is akin to getting lost in a labyrinth\u2014it may appear that it’s leading you somewhere<\/em>, but such pointless wandering can seldom lead you to your desired destination.<\/p>\n\n\n\n

Whether it’s reaching your destination or hitting a conversion goal on your website, the process involves numerous baby steps that you take in the form of iterative testing<\/a> and hypothecation. Along with the process, they keep evolving to find an optimal route to desired conversions. <\/p>\n\n\n\n

Download Free: A\/B Testing Guide<\/span><\/a><\/em><\/h2>\n\n\n\n

In this post, you will learn about constructing strong A\/B testing hypotheses<\/a>, which will ensure you are moving in the right direction to reach your destination in time.<\/p>\n\n\n

What is a hypothesis?<\/strong><\/strong><\/h2>\n\n\n

A hypothesis is a proposed explanation or solution to a problem. Think of it as a glue that ties the problem to a solution. For instance, you could hypothesize that adding trust badges to your payment page could cater to the problem of low conversion rates on that page. As you\u2019d notice, the hypothesis is made up of two variables, namely the cause (the action we want to test) and effect (the outcome we expect).<\/p>\n\n\n\n

One day you wake up and want to run a test for the color of the CTA button<\/a> on your website. You rush and achieve conversion lifts with this change, but you do not know exactly what you wanted from this test? <\/span><\/p>\n\n\n\n

You did not pay any heed to your <\/span>conversion funnel<\/span><\/a>. That is, you did not analyze the test fully before executing it! <\/span><\/p>\n\n\n\n

\"Blog<\/a><\/figure>\n\n\n\n

There is a difference between running a test scientifically to prove something<\/em> and generating random results.Now, this something<\/em> is your hypothesis.<\/p>\n\n\n

Components of a hypothesis<\/strong><\/strong><\/h2>\n\n\n

As you would notice here, in the world of user experience optimization<\/a>, a strong hypothesis is made up of three components: defining a problem, describing a proposed solution, and measuring results.<\/p>\n\n\n

For instance, you could hypothesize that adding trust badges to your payment page could fix <\/span>low conversion rates<\/span><\/a> on that page. You can find out why that happens by identifying the right metrics for success.<\/span><\/p>\n

But how does one begin to start formulating a hypothesis?<\/span><\/p>\n\n

How to formulate a winning hypothesis?<\/strong><\/strong><\/h2>\n\n

It is not wise to blindly follow best practices crafted by someone else for your business. Every business is unique, so are its strategies. It is always recommended to construct your own best practices.\u00a0<\/span><\/p>\n

Below are some essential elements that make a solid hypothesis:<\/span><\/p>\n\n

1. They aim to alter customer behavior, either positively or negatively.<\/strong><\/strong><\/h3>\n\n

A psychological principle often forms the basis of a hypothesis that triggers a reaction from prospects.\u00a0<\/span><\/p>\n

It changes the way they perceive your offer\/brand, which alters their behavior during the test. Like in the headline example below, the urgency of the new message is why the variation headline is expected to perform better than the original headline.<\/span><\/p>\n

Changing the headline from \u2018Grab your tickets now!\u2019 to\u00a0<\/span><\/p>\n

\u2018Tickets filling out soon \u2013 only last 50 left!\u2019 could increase ticket sales online.\u00a0<\/span><\/p>\n

Only because you follow the above syntax to formulate a hypothesis doesn\u2019t mean that you\u2019ve got the winning hypothesis.<\/span><\/p>\n\n

2. They focus on deriving customer learning from tests.<\/strong><\/strong><\/h3>\n\n

When pros develop a hypothesis<\/a>, their focus is on the big picture.\u00a0<\/span><\/p>\n\n\n

So your test might give you any result. What’s important is for you to analyze \u2018why\u2019 your visitors behaved in a certain way? <\/p>\n\n\n\n

As data-driven marketers, it might seem difficult sometimes to make your peace with negative lifts. <\/p>\n\n\n\n

But if the test reveals significant customer learning, it can pave the way for colossal conversion lifts in the future. <\/p>\n\n\n\n

For example, when Michael Aagaard, the conversion copywriter of Content Verve, conducted a CTA copy test on his client\u2019s website. He learned that changing the CTA copy from \u2018Start your 30 day free trial\u2019 to \u2018Start my 30-day free trial\u2019 resulted in a 90% increase in sign-ups.<\/p>\n\n\n\n

\"screenshot
Image source: Marketing Experiments<\/a><\/figcaption><\/figure>\n\n\n\n

However, he cautioned that applying this technique would not work across the linguistic globe though.<\/p>\n\n\n

3. They are derived from at least some evidence.<\/strong><\/strong><\/h3>\n\n\n

You can explore many avenues to construct good hypotheses. <\/p>\n\n\n\n

The image below by the popular Conversion Expert, Craig Sullivan, shows several ways to collect meaningful insights.<\/p>\n\n\n

\n
\"ways
Image source: Slideshare<\/a>
<\/figcaption><\/figure>\n<\/div>\n\n\n

a. Usability testing<\/strong><\/h4>\n\n\n\n

In simple words, you can sit and observe how your test participants use your website. <\/p>\n\n\n\n

Make notes where they get stuck or confused. Form your hypotheses to improve these situations and let A\/B tests decide if they work for you.<\/p>\n\n\n\n

This method gives you exceptional insights into your customers\u2019 workarounds and struggles in using your website. <\/p>\n\n\n\n

Write down the exact questions you would like to ask during your research. Asking questions related to the homepage, checkout, pricing page, and navigation reveal great insights. <\/p>\n\n\n\n

Some sample questions you can ask are: <\/p>\n\n\n\n