Chance to beat original is the probability that a variation will perform better than the control version of your website or mobile app. For example, a CTBO of 60% means that the variation is likely to perform better than the control 60% of the time.

**Attention!**

Currently, this method of calculating the CTBO is only used for Multivariate and Personalization campaigns in VWO, as they still follow the Frequentist method of calculating a winner. To determine the winning variation in A/B and Split URL Campaigns in VWO, we follow the SmartStats (Bayesian) method of calculating the chance to beat all versions.

The success or failure of a variation is calculated by comparing the results against a base version (by default, the control page). You can change the base for your campaign at any time.

Please note that VWO calculates the CTBO on the basis of campaign threshold settings you have defined in **Settings > Campaign Settings** as displayed in the following screenshot:

To determine CTBO of a variation, VWO calculates the following values:

- Conversion rates for control and variation
- Standard error for control and variation
- CTBO of a variation

### Calculate conversion rates of the control and variations

Mathematically, conversion rate is a binomial random variable, meaning it can have two possible values. It is represented as the variable p. In A/B testing, we calculate the value of p by observing n visitors on the webpage that is being tested. From those n visitors, we then calculate the number of conversions. The percentage value that results is the conversion rate of your website or app.

*Conversion rate (p) = actual conversion/visits to the website*

The percentage value is the conversion rate of your website.

For example, let us say you are running a campaign to test the new design of your landing page. The Control receives 894 visits, of which 423 converted. On the other hand, the variation received 863 visits, of which 458 converted. This implies that the conversion percentage value of the control is 47%, while that of the variation is 53%.

### Calculate standard error for the control and variations

If you repeat the same test multiple times, it is likely that you will get a slightly different value of p (or conversion rate of a variation) every time. This happens because in statistical terms, we’re sampling and like every sample, there’s a range of error associated with it (because it does not cover the entire population). To avoid doing repeated tests, we can use already established statistical formulas calculate the standard error to determine how much deviation from average conversion rate (p) can be expected from the test results. The deviation is a range within which the conversion rate is usually found. Smaller the deviation, more confident you can be about estimating true conversion rate. For a given conversion rate (p) and number of trials (n), standard error is calculated as:

*Standard Error (SE) = Square root of (p * (1-p) / n)*

In our example, the SE for control = Square root of (.4732*(1-.473/894) = 1.67

And, the SE for variation = Square root of (.5307*(1-.5307/863) =1.70

(Please note that the percentage value are converted to decimal in the above calculations).

To get 95% range for conversion rate, multiply the standard error value by 2 (or 1.96 to be precise). You can be sure with 95% confidence that your true conversion rate lies within this range: p % ± 2 * SE.

In VWO, we calculate conversion rate range for 80%, not 95%. Hence, we multiply the SE value by 1.28.

In the example discussed above, the standard error for the control is 1.67, while that of the variation is 1.70. Next, multiply the values by 1.28. The conversion rate range thus calculated is ± 2.14% for control and ± 2.17% for the variation.

### Calculate CTBO for a variation

To determine CTBO for a variation, we calculate the Z-score for control and variation. A z-score is used in statistics to model any normal distribution as a standard normal distribution.

Where,

- Pv= Conversion rate of variation
- Pc= Conversion rate of control
- SEv= Standard error of variation
- SEc= Standard error of control

For our example discussed above, the z-score can be derived as follows:

** Z-score = (0.53 - 0.47)/Square root of 0.01698882 +0.0166982** = 2.41605

After the z-score is calculated, VWO calculates standard normal distribution to derive the chance to beat original percentage for the variation. You can refer to the following normal distribution table to understand how CTBO is derived for a variation:

As we see, for our z-score of 2.41, the normal distribution value is approximately 99%.

To make the calculation easier for you, VWO provides a campaign significance calculator. Click here to download an excel spreadsheet to help you determine the statistical significance for your campaign.

VWO does not calculate CTBO for the following scenarios:

- The conversion rate of control is 0 or 1
- The conversion rate of variation is 0 or 1
- Visitors in the variation is less than the set threshold value (sample size)
- Standard Error of either control or variation is 0

**Note:** You can edit CTBO threshold values and sample size using VWO dashboard under **Settings > Campaign Settings**.