How to Read a Test Report

Last updated

Deriving meaningful insights from your tests and applying them to improve your website’s conversion rate is the ultimate objective of your tests. The reports help you to validate the hypothesis you are testing and derive a better understanding of visitor behavior on your website.

To access the report section of a test, on the navigation bar, select the test type under Test. From the list of tests, select the test for which you want to view the report.ab-preview1

Viewing Reports

On the summary page of the test you have selected, click the Detailed Report tab to view individual reports for all the goals and variations in the test.

  • To view a specific goal’s report, select the goal from all the goals listed on the right side of the dashboard.ab-preview8
  • To view report for variations, click the Variations tab under Detailed Report. Next, select a variation from all the variations listed on the right side of the dashboard.ab-preview9

The reports are represented graphically as well as numerically. Both the reports type have several common parameters.

Graphical Reports

  • Conversion rates: The percentage of unique visitors who took the desired action and triggered the goal.
  • Conversions: The number of unique visitors who triggered the goal.
  • Visitors: The total number of unique visitors who visited the test page.
  • Revenue: The revenue generated for the goal. This filter works only if you have defined a revenue goal in the test.
  • Date Range: The date range for which you want to view the reports. You can select the date range or view day-wise or cumulative reports.
  • Segment: If you have targeted a specific visitor segment, select the segment for which you want to view reports.

Numerical Reports

  • Revenue Per Visitor: The average revenue generated by each visitor. This value is expressed as a range.
  • Improvement: The improvement in the conversion percentage of a variation as compared to the conversion percentage of the Control. This value is expressed as a range.
  • Potential Loss: The lift you can lose if you deployed the A version of the website as the winner when in fact B was actually better.
  • Chance to Beat Control (CTBO): The probability that a variation will perform better than the control version of your website or mobile app. For example, a CTBO of 60% means that the variation is likely to perform better than the control 60% of the time, or there is a 40% chance that your results are completely due to a random chance. To know more about how CTBO is calculated, click here.
  • Revenue Per Conversion: The average revenue generated for every visitor converted.
  • Conversion/Visitors: The ratio of the number of conversions in a variation to the number of visits.

Parameters exclusive to the Variations Report

  • Variation: All variations you have created for the test
  • Conversion Rate Range: The range within which the actual conversion rate lies, taking into consideration the margin of error you are willing to accept. For example, if the conversion rates range for a variation is 8.23% (+-2%), then the conversion range of the variation would be between 6.23% – 10.23%.

Contact Us / Login

Product
Resources Home