This website works best with JavaScript enabled. Learn how to enable JavaScript.

Best Practices for Creating an A/B Test

Last updated

VWO is a tried and tested tool delivering the best possible website experience to your website visitors. However, there are a few best practices that will help you achieve optimal performance from VWO. Creating an A/B test using VWO involves the following stages. Taking care of a few important things at every stage will help you get the maximum out of your website testing plans:

  1. Add test URLs
  2. Create test variations
  3. Add goals
  4. Set prior values and test expectations

Adding Test URLs

  • By default, the Simple mode is selected in the URLs. You must add only an “exact URL match” in this mode. To add a URL pattern or a regex to run the test on multiple pages of the website, select the Advanced mode.best practices on AB test
  • Under the Advanced mode, you can select different URL types to add patterns, regex, keywords, or match cases to run a test across multiple URLs. You can also click the Include URL or Exclude URL options to add multiple pages to the test plan.advanced URL settings for AB test
  • In the Enter URL of the page you want to preview field, enter a default campaign URL, which must be an exact URL. This is the page which loads as the initial URL in the editor, heatmaps, and previews. If you add a pattern or regex in this field, the page will not load in the editor or the previews will not open.preview url for AB tests

Creating Test Variations by Using Editor

  • Do not use the Edit HTML operation on dynamic elements. For example, if you try using the editor to change the dynamic pricing of items added to the cart, the price will appear to be static in the variations created.
  • Do not use the Move or Resize operation on responsive websites, because doing so may break its responsiveness. You can add media queries or custom JS code if you want to apply changes for different devices of various dimensions, widths, and heights.
  • Wrap the changes in setInterval() or use the _vis_opt_element_loaded() function whenever the elements load with delay on the page.
  • Do not make changes to the variation while the campaign is running. Always clone the campaign first and run it again, because the changes might affect the data already collected for the test.
  • Preview all your tests pages after making any change. You can keep the Previews tab opened in a new tab or window to view the changes in real time. This helps ensure that your test variations don’t appear broken when making multiple changes on dependent elements.

To learn more about Editor Best Practices, click here.

Adding Goals

To learn about the different types of goals you can use during your VWO test, click here. When adding goals, make a note of the following points:

  • Do not change the existing primary goal, because such a change can skew the collected data.
  • If you set up the goal URL on the same test page URL, you will notice a conversion on page refresh. VWO will record a conversion as soon as a visitor lands on the Goal URL and only looks for the page visit cookie. Such reports are insignificant and may not serve the real purpose of your test.
  • You should track goal conversions on all existing variations. If you set up a goal for only one variation, you will not be able to study or compare data between variations to derive better insights from the test.
  • Use live previews to verify if the goals are working fine. To learn how to preview goals, click here.
  • If you want to track goals on a page which is neither a campaign URL nor a goal URL, you must set up a dummy ‘track page visit’ goal for the URL to allow VWO to send the test settings to the page.

For example, the campaign is running on http://abc.com. A link needs to be tracked; however, the link is present on the http://abc.com/test.html page. So, a “track page visit” goal needs to be created first for the http://abc.com/test.html URL to receive the campaign settings and then start the goal tracking process.

Prior Values and Test Expectations

  • To ensure accurate reports and to save time while getting the test result, define the prior values correctly.
  • If the average number of monthly visitors defined in the priors is low compared to the actual visitors tracked during the test, you may see a warning saying “traffic is too low for the campaign to run.”
  • You must not change the prior values in the middle of a test. If you change values while the campaign is running, it can delay the result, because VWO will need to adjust the test settings per the new prior values.
  • The value you enter for “minimum lift in conversion” is relative to the actual conversion rate. For example, if the existing conversion rate is 5% and you want the conversion rate to be 10% for the variation, the minimum lift field will have the value as 100% (relative to the existing rate).

The last step in the A/B test creation process is finalizing the test settings for traffic allocation, third-party integrations, and other options. You can select the options relevant to your test requirements, and then start running the test.

Was this article helpful?

Thanks for your feedback!
Please email us at support@vwo.com to tell us what you were looking for. We'll send the answer to your inbox.

Contact Us / Login