Numerous times our clients and customers ask: how many visitors do I need to test in order to get statistically significant results in an A/B test? Unfortunately, there is no straightforward answer to this question as it depends on a number of factors. In the past, we have provided you with an A/B test duration calculator in Excel and also provided with an online tool to calculate the same. However, how it is calculated is sort of a black magic.
In following presentation, I describe just what exactly happens when we calculate number of visitors needed for an A/B test. As the presentation aims to be a high level overview, I won’t be delving into mathematics of it.
Hope you like the presentation! Please let me know if you have any questions or need clarifications.