{"id":7675,"date":"2013-09-03T19:00:53","date_gmt":"2013-09-03T13:30:53","guid":{"rendered":"https:\/\/vwo.com\/blog\/?p=7675"},"modified":"2025-04-30T16:13:18","modified_gmt":"2025-04-30T10:43:18","slug":"ab-test-big-changes","status":"publish","type":"post","link":"https:\/\/vwo.com\/blog\/ab-test-big-changes\/","title":{"rendered":"A\/B Testing for Big Wins – When You Should Do It & How You Should Do It"},"content":{"rendered":"\n

Button color tests. Font size tests. Headline tests. Run them all you want.<\/p>\n\n\n\n

But remember that your website\u2019s conversion rate<\/a> is only as limited as the risks you take. Small tweaks = small wins. If you’re craving for big wins, you’ll have to make big changes.<\/p>\n\n\n\n

I understand that you want to \u201cplay it safe\u201d and be cautious. But sometimes you should test a radical website redesign against your original one<\/a>, instead of testing small tweaks over and again.<\/p>\n\n\n\n

Download Free: A\/B Testing Guide<\/em><\/span><\/a><\/h3>\n\n\n

Scenarios When a Radical Redesign Makes More Sense than Testing Small Tweaks:<\/h2>\n\n

When You Have a Low-traffic Site<\/h3>\n\n\n

The problem with a low-traffic site is, that small tweaks will take a long time to reach the 95% confidence level for a valid test result. But if you test two radically different designs against each other, you will require a smaller sample size and the conclusive results will be reached in much less duration.<\/p>\n\n\n\n

You can see the table below elaborating on this, where the test duration<\/a> has been calculated from our Split test duration calculator<\/a>:<\/p>\n\n\n\n

Assumptions:<\/strong>
Number of visitors: 500 per day
The current conversion rate of the site: is 2%
Variations: 2<\/p>\n\n\n\n

Percentage Increase<\/th>Split Test Duration<\/th><\/tr>
5%<\/td>1254 Days<\/td><\/tr>
10%<\/td>314 Days<\/td><\/tr>
15%<\/td>139 Days<\/td><\/tr>
25%<\/td>50 Days<\/td><\/tr>
50%<\/td>13 Days<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

Assumptions:<\/strong>
Number of visitors: 20,000 per day
The current conversion rate of the site: is 2%
Variations: 2<\/p>\n\n\n\n

Percentage Increase<\/th>Split Test Duration<\/th><\/tr>
5%<\/td>31 Days<\/td><\/tr>
10%<\/td>8 Days<\/td><\/tr>
15%<\/td>3 Days<\/td><\/tr>
25%<\/td>1 Day<\/td><\/tr>
50%<\/td>Less than 1 day<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

As you can see in the above tables, the higher the expected increase in the conversion rate (which also reflects how big the change you made), the lesser the duration for which you will have to run the test.<\/p>\n\n\n\n

The only difference between the two tables is the number of daily visitors received by the sites.<\/p>\n\n\n\n

This clearly shows that a high-traffic site can test small changes as much as it wants. But for sites that receive less traffic, it is best to make big changes (at least to start with). After this, you can tweak your new, winner-design and optimize it further for small wins.<\/p>\n\n\n

When No Matter What Tweak You Test, You Get Negative Results<\/h3>\n\n\n

When you focus on small tests only and test every element that you can find on the page, there will come a time when any tweak that you make will only give you negative results. This means that your current design has reached its maximum conversion potential, also known as the “local maximum.”<\/p>\n\n\n\n

Solution? Test your fully-optimized page against a new design that is entirely different from your current design.<\/p>\n\n\n

When Your Current Design Has Huge Scope of Improvement<\/h3>\n\n\n

If you’re a beginner in A\/B testing<\/a>, it’s perfectly fine that you start with 2-3 small tests first because you’re testing the waters. And if you’re someone who needs to convince your management of the power of AB testing, small tests are again your best bet.<\/p>\n\n\n\n

But if you are well aware of conversion optimization practices and see a huge scope for improvement in the current website design, be bold and start with testing a different design at once.<\/p>\n\n\n\n

Wasting time on testing smaller tweaks on the current design that needs a complete overhaul is a mere waste of time.<\/p>\n\n\n

When You are Not Ready to Settle for Less and You Want Big Wins<\/h3>\n\n\n

Small tweaks can only take you so far. But if you hear about huge wins and wish that you will hit the jackpot, it’s time you leave the “safe harbor” of running small tests and testing some radical redesigns.<\/p>\n\n\n\n

This involves high risk, yes. But if you do it right, your chances of hitting your jackpot will also be much higher.<\/p>\n\n\n\n

But wait…” How exactly can I do it right?” Is that what you’re thinking? Read the approaches given below and you will hopefully find your way to a great hypothesis.<\/p>\n\n\n

Approaches to Come Up with a Solid Hypothesis for a High-Converting Redesign<\/h2>\n\n

1. Take Them Closer to the Must-have Experience<\/h3>\n\n\n

The must-have experience is allowing visitors to understand the true value of the product by trying it out themselves.<\/p>\n\n\n\n

Reduce the number of steps or even the number of clicks that a user must complete to try out your product. If your product is powerful, making people realize how useful your product is, can do wonders for your conversions.<\/p>\n\n\n\n

Eliminate any complicated steps or requirements, unless necessary. The less effort is required by the user, the better it is.<\/p>\n\n\n\n

Hello, Bar understands this perfectly well and guides visitors to try out their product with a single click on their homepage. Here’s what their visitors see on the next page:<\/p>\n\n\n

\n
\"Hello
Image Source: Hellobar<\/figcaption><\/figure>\n<\/div>\n\n\n

Notice how the fields above do not ask for any personal information of the user at all. Visitors can try out their product right away without giving any email address, or name, or filling out signup forms.<\/p>\n\n\n\n

The point is, that your conversion goal should come after the must-have experience and not before it.<\/strong><\/p>\n\n\n\n

It might seem like this approach is only fit for SaaS websites<\/a>, but that’s not true. See the “Click to look inside” option provided by Amazon:<\/p>\n\n\n

\n
\"Amazon
Image Source: Amazon.com<\/figcaption><\/figure>\n<\/div>\n\n\n

If the nature of your business does not allow you to provide this must-have experience on your website, you can treat your final order or sale as your must-have experience.<\/p>\n\n\n\n

Adding a sign-up goal before their must-have experience (final sale), may make you lose tons of money. This is why guest checkouts are so popular these days. And how can you forget the awesome 1-click checkout of Amazon?<\/p>\n\n\n\n

Start from your conversion funnel in web analytics. Lookout for steps where you are getting the most drop-offs.<\/p>\n\n\n\n

Consider if you can completely skip any of these steps, or at least reduce the friction for visitors by reducing the number of clicks or form fields that are in their way in getting closer to the must-have experience.<\/p>\n\n\n

2. Challenge the Approach of the Current Design<\/h3>\n\n\n

Testing a radical makeover doesn’t mean that you design a page randomly that is drastically different from your current website design.<\/p>\n\n\n\n

The point here is to focus on challenging the assumptions or approaches of the current design and testing it based on a solid hypothesis.<\/p>\n\n\n\n

For example, Sierra Tucson is a rehabilitation facility that tested a radical redesign on its web page. They found that their trust-focus landing page converted better<\/a> than their luxury-inclined page. This got them 220% more leads.<\/p>\n\n\n\n

Download Free: A\/B Testing Guide<\/em><\/span><\/a><\/h3>\n\n\n

3. Conduct Customer Surveys<\/h3>\n\n\n

You cannot be your customer. To understand what special points about your product or service resonate best with your audience or if they have any concerns or apprehensions, conducting customer surveys can provide great insights.<\/p>\n\n\n\n

Once you know what your customers like the most about you, you can emphasize it on your page to get positive responses. You can combine this with heatmaps\/clickmaps in your VWO app<\/a> to see what interests your audience the most and redesign your page accordingly.<\/p>\n\n\n\n

One of our customers recently conducted a customer survey, followed by testing a radical redesign against their original page. The insights from the customer survey were used to design almost every element on the redesigned page along with the changes in the copy that used the same words that their customers used in the survey to define the core benefit of their service.<\/p>\n\n\n\n

The redesigned page increased their sales<\/a> by 64.8%. You can read the complete case study here.<\/a><\/p>\n\n\n\n

Also, you can watch this webinar on how to understand customers with VWO Surveys:<\/span><\/p>\n\n\n\n

\n