{"id":60,"date":"2010-01-26T16:26:47","date_gmt":"2010-01-26T16:26:47","guid":{"rendered":"https:\/\/vwo.com\/blog\/?p=60"},"modified":"2025-04-30T16:20:57","modified_gmt":"2025-04-30T10:50:57","slug":"what-you-really-need-to-know-about-mathematics-of-ab-split-testing","status":"publish","type":"post","link":"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/","title":{"rendered":"What You Really Need To Know About The Mathematics Of A\/B Split Testing"},"content":{"rendered":"\n<p>Recently, I published an A\/B <a href=\"https:\/\/vwo.com\/testing\/split-url-testing\/\">split testing<\/a> case study where an <a href=\"https:\/\/vwo.com\/success-stories\/medaliaart\/\">eCommerce store reduced the bounce rate by 20%<\/a>. Some of the blog readers were worried about the <a href=\"https:\/\/vwo.com\/glossary\/statistical-significance\/\">statistical significance<\/a> of the results. Their main concern was that a value of 125-150 visitors per variation is not enough to produce reliable results. This concern is a typical by-product of having superficial knowledge of statistics which powers A\/B (and multivariate) testing. I\u2019m writing this post to provide an essential primer on the mathematics of A\/B <a href=\"https:\/\/vwo.com\/blog\/split-testing\/\">split testing<\/a> so that you never jump to a conclusion on the reliability of test results simply on the basis of the number of visitors.<\/p>\n\n\n\n<h2 class=\"wp-block-heading has-text-align-center\"><em><a href=\"https:\/\/vwo.com\/ab-testing\/#guide-download\"><span style=\"text-decoration: underline\">Download Free: A\/B Testing Guide<\/span><\/a><\/em><\/h2>\n\n\n<h2 class=\"js-cro-guide-subheading gtm_heading \" data-level=\"level1\" data-menu=\"What exactly goes behind A\/B split testing?\" id=\"what-exactly-goes-behind-a-b-split-testing\" data-menu-id=\"what-exactly-goes-behind-a-b-split-testing\" style=\"text-align:left\"><strong>What exactly goes behind A\/B split testing?<\/strong><\/h2>\n\n\n<p>Imagine your website as a black box containing balls of two colors (red and green) in unequal proportions. Every time a visitor arrives on your website, he takes out a ball from that box: if it is green, he makes a purchase. If the ball is red, he leaves the website. This way, essentially, that black box decides the <a href=\"https:\/\/vwo.com\/glossary\/conversion-rate\/\">conversion rate<\/a> of your website.<\/p>\n\n\n\n<p>A key point to note here is that you cannot look inside the box to count the number of balls of different colors to determine the true conversion rate. You can only <em>estimate<\/em> the conversion rate based on different balls you see coming out of that box. Because conversion rate is an estimate (or a guess), you always have a range for it, never a single value. For example, mathematically, the way you describe a range is:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>&#8220;Based on the information I have, 95% of the times conversion rate of my website ranges from 4.5%-7%.&#8221;<\/p>\n<\/blockquote>\n\n\n\n<p>As you would expect, with more visitors, you get to observe more balls. Hence, your range gets narrower, and your estimate starts approaching the true conversion rate.<\/p>\n\n\n<h2 class=\"js-cro-guide-subheading gtm_heading \" data-level=\"level1\" data-menu=\"The maths of A\/B split testing\" id=\"the-maths-of-a-b-split-testing\" data-menu-id=\"the-maths-of-a-b-split-testing\" style=\"text-align:left\"><strong>The maths of A\/B split testing<\/strong><\/h2>\n\n\n<p>Mathematically, <a href=\"https:\/\/en.wikipedia.org\/wiki\/Binomial_distribution\" target=\"_blank\" rel=\"noreferrer noopener\">the conversion rate is represented by a binomial random variable<\/a>, which is a fancy way of saying that it can have two possible values: conversion or non-conversion. Let\u2019s call this variable <em>p<\/em>. Our job is to estimate the value of <em>p, <\/em>and for that, we do <em>n<\/em> trials (or observe <em>n<\/em> visits to the website). After observing those <em>n<\/em> visits, we calculate how many visits resulted in a conversion. That percentage value (which we represent from 0 to 1 instead of 0% to 100%) is the conversion rate of your website.<\/p>\n\n\n\n<p>Now imagine that you repeat this experiment multiple times. It is very likely that, due to chance, every single time, you will calculate a different value of <em>p<\/em>. Having all (different) values of <em>p<\/em>, you get a range for the conversion rate (which is what we want for the next step of analysis). To avoid doing repeated experiments, statistics has a neat trick in its toolbox.&nbsp;There is a concept called <em>standard error<\/em>, which tells how much deviation from the average conversion rate (<em>p<\/em>) can be expected if this experiment is repeated multiple times. The smaller the deviation, the more confident you can be about estimating the true conversion rate. For a given conversion rate (<em>p<\/em>) and the number of trials (<em>n<\/em>), the standard error is calculated as:<\/p>\n\n\n\n<p><em>Standard Error (SE) = Square root of (p * (1-p) \/ n)<\/em><\/p>\n\n\n\n<p>Without going much into details, to get a 95% range for conversion rate multiply the standard error value by 2 (or 1.96 to be precise). In other words, you can be sure with 95% confidence that your true conversion rate lies within this range: <em>p<\/em> % \u00b1 2 * <em>SE<\/em><\/p>\n\n\n\n<p>(In VWO, when we show the conversion rate range in reports, we show it for 80%, not 95%. So we multiply standard error by 1.28).<\/p>\n\n\n\n<p>Apart from standard error, <a href=\"https:\/\/vwo.com\/blog\/errors-in-ab-testing\/\">while doing A\/B testing, you would have to take into consideration Type I &amp; Type II errors<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading has-text-align-center\"><em><a href=\"https:\/\/vwo.com\/ab-testing\/#guide-download\"><span style=\"text-decoration: underline\">Download Free: A\/B Testing Guide<\/span><\/a><\/em><\/h2>\n\n\n<h2 class=\"js-cro-guide-subheading gtm_heading \" data-level=\"level1\" data-menu=\"What does it have to do with reliability of results?\" id=\"what-does-it-have-to-do-with-reliability-of-results\" data-menu-id=\"what-does-it-have-to-do-with-reliability-of-results\" style=\"text-align:left\"><strong>What does it have to do with reliability of results?<\/strong><\/h2>\n\n\n<p>In addition to calculating the conversion rate of the website, we also calculate a range for its variations in an A\/B split test. Because we have already established (with 95% confidence) that the true conversion rate lies within that range, all we have to observe now is the overlap between the conversion rate range of the website (control) and its variation. If there is no overlap, the variation is definitely better (or worse if the variation has a lower conversion rate) than the control. It is that simple.<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p><em>As an example, suppose control conversion rate has a range of&nbsp;6.5% \u00b1 1.5% and a variation has a range of 9% \u00b1 1%. In this case, there is no overlap, and you can be sure about the reliability of the results<\/em>.<\/p>\n<\/blockquote>\n\n\n<h2 class=\"js-cro-guide-subheading gtm_heading \" data-level=\"level1\" data-menu=\"Do you call all that math simple?\" id=\"do-you-call-all-that-math-simple\" data-menu-id=\"do-you-call-all-that-math-simple\" style=\"text-align:left\"><strong>Do you call all that math simple?<\/strong><\/h2>\n\n\n<p>Okay, not really simple, but it is definitely intuitive. To save the trouble of doing all the math by yourself, either use a tool like <a href=\"https:\/\/vwo.com\/testing\/ab-testing\/\">VWO Testing<\/a> which automatically does all the number crunching for you. Or, if you are doing a test manually (such as for Adwords), use our <a href=\"https:\/\/vwo.com\/tools\/ab-test-significance-calculator\/\">free A\/B split test significance calculator<\/a>.<\/p>\n\n\n<h2 class=\"js-cro-guide-subheading gtm_heading \" data-level=\"level1\" data-menu=\"So, what is the take-home lesson here?\" id=\"so-what-is-the-take-home-lesson-here\" data-menu-id=\"so-what-is-the-take-home-lesson-here\" style=\"text-align:left\"><strong>So, what is the take-home lesson here?<\/strong><\/h2>\n\n\n<p><em>Always, always, always<\/em> use an A\/B split testing calculator to determine the significance of results before jumping to conclusions. Sometimes you may discount significant results as non-significant solely on the basis of the number of visitors. Sometimes you may think results are significant due to the large number of visitors when in fact they are not.<\/p>\n\n\n\n<p>You really want to avoid both scenarios, don\u2019t you?<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><a href=\"#free-trial\"><img loading=\"lazy\" decoding=\"async\" width=\"2400\" height=\"1256\" src=\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2021\/09\/End-Banner_What-you-really-need-to-know-about-mathematics-of-AB-split-testing.jpg\" alt=\"End Banner VWO Split Testing\" class=\"wp-image-60859\" srcset=\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2021\/09\/End-Banner_What-you-really-need-to-know-about-mathematics-of-AB-split-testing.jpg 2400w, https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2021\/09\/End-Banner_What-you-really-need-to-know-about-mathematics-of-AB-split-testing.jpg?tr=w-1600 1600w, https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2021\/09\/End-Banner_What-you-really-need-to-know-about-mathematics-of-AB-split-testing.jpg?tr=w-1366 1366w, https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2021\/09\/End-Banner_What-you-really-need-to-know-about-mathematics-of-AB-split-testing.jpg?tr=w-1024 1024w, https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2021\/09\/End-Banner_What-you-really-need-to-know-about-mathematics-of-AB-split-testing.jpg?tr=w-768 768w, https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2021\/09\/End-Banner_What-you-really-need-to-know-about-mathematics-of-AB-split-testing.jpg?tr=w-640 640w, https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2021\/09\/End-Banner_What-you-really-need-to-know-about-mathematics-of-AB-split-testing.jpg?tr=w-375 375w\" sizes=\"(max-width: 2400px) 100vw, 2400px\" \/><\/a><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>Recently, I published an A\/B split testing case study where an eCommerce store reduced the bounce rate by 20%. Some of the blog readers were worried about the statistical significance of the results. Their main concern was that a value of 125-150 visitors per variation is not enough to produce reliable results. This concern is&#8230;<\/p>\n","protected":false},"author":14,"featured_media":56806,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"post_read_time":4,"footnotes":""},"categories":[10558],"tags":[],"feature":[1852,10526],"industry-type":[],"product":[10626],"role":[10632,10641,10638],"region":[],"class_list":["post-60","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-calculator","feature-ab-testing","feature-experimentation-platform"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.9 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>A\/B Split Testing: Everything You Need To Know - VWO<\/title>\n<meta name=\"description\" content=\"An essential primer on mathematics of a\/b split testing so that you never conclude the reliability of test results simply on basis of number of visitors.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"A\/B Split Testing: Everything You Need To Know - VWO\" \/>\n<meta property=\"og:description\" content=\"An essential primer on mathematics of a\/b split testing so that you never conclude the reliability of test results simply on basis of number of visitors.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/\" \/>\n<meta property=\"og:site_name\" content=\"Blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/vwoofficial\/\" \/>\n<meta property=\"article:published_time\" content=\"2010-01-26T16:26:47+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-30T10:50:57+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2010\/01\/OG-image_What-you-really-need-to-know-about-mathematics-of-AB-split-testing.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"630\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Paras Chopra\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@paraschopra\" \/>\n<meta name=\"twitter:site\" content=\"@VWO\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Paras Chopra\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/\"},\"author\":{\"name\":\"Paras Chopra\",\"@id\":\"https:\/\/vwo.com\/blog\/#\/schema\/person\/6a4fd0d62df48bca6a31f484f7c4e5b6\"},\"headline\":\"What You Really Need To Know About The Mathematics Of A\/B Split Testing\",\"datePublished\":\"2010-01-26T16:26:47+00:00\",\"dateModified\":\"2025-04-30T10:50:57+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/\"},\"wordCount\":924,\"publisher\":{\"@id\":\"https:\/\/vwo.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2010\/01\/Feature-image_What-you-really-need-to-know-about-mathematics-of-AB-split-testing.png\",\"articleSection\":[\"Calculator\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/\",\"url\":\"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/\",\"name\":\"A\/B Split Testing: Everything You Need To Know - VWO\",\"isPartOf\":{\"@id\":\"https:\/\/vwo.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2010\/01\/Feature-image_What-you-really-need-to-know-about-mathematics-of-AB-split-testing.png\",\"datePublished\":\"2010-01-26T16:26:47+00:00\",\"dateModified\":\"2025-04-30T10:50:57+00:00\",\"description\":\"An essential primer on mathematics of a\/b split testing so that you never conclude the reliability of test results simply on basis of number of visitors.\",\"breadcrumb\":{\"@id\":\"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/#primaryimage\",\"url\":\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2010\/01\/Feature-image_What-you-really-need-to-know-about-mathematics-of-AB-split-testing.png\",\"contentUrl\":\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2010\/01\/Feature-image_What-you-really-need-to-know-about-mathematics-of-AB-split-testing.png\",\"width\":1200,\"height\":700,\"caption\":\"What You Really Need To Know About Mathematics Of Ab Split Testing\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/vwo.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Calculator\",\"item\":\"https:\/\/vwo.com\/blog\/calculator\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"What You Really Need To Know About The Mathematics Of A\/B Split Testing\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/vwo.com\/blog\/#website\",\"url\":\"https:\/\/vwo.com\/blog\/\",\"name\":\"Blog\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/vwo.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/vwo.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/vwo.com\/blog\/#organization\",\"name\":\"VWO\",\"url\":\"https:\/\/vwo.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/vwo.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2018\/09\/VWOLogo.png\",\"contentUrl\":\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2018\/09\/VWOLogo.png\",\"width\":780,\"height\":492,\"caption\":\"VWO\"},\"image\":{\"@id\":\"https:\/\/vwo.com\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/vwoofficial\/\",\"https:\/\/x.com\/VWO\",\"https:\/\/www.instagram.com\/vwoofficial\/\",\"https:\/\/www.linkedin.com\/company\/vwo\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/vwo.com\/blog\/#\/schema\/person\/6a4fd0d62df48bca6a31f484f7c4e5b6\",\"name\":\"Paras Chopra\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/vwo.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2019\/04\/paras-150x150.jpg\",\"contentUrl\":\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2019\/04\/paras-150x150.jpg\",\"caption\":\"Paras Chopra\"},\"description\":\"I started Wingify in early 2009 to enable businesses to design and deploy great customer experiences for their websites and apps. I have a background in machine learning and am a gold medalist from Delhi College of Engineering. I have been featured twice in the Forbes 30 under 30 list - India and Asia. I'm an entrepreneur by profession and my curiosity is wide-ranging. Follow me at @paraschopra on Twitter. You can email me at paras@wingify.com\",\"sameAs\":[\"https:\/\/invertedpassion.com\/\",\"https:\/\/in.linkedin.com\/in\/paraschopra\",\"https:\/\/x.com\/paraschopra\"],\"url\":\"https:\/\/vwo.com\/blog\/author\/paraschopra\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"A\/B Split Testing: Everything You Need To Know - VWO","description":"An essential primer on mathematics of a\/b split testing so that you never conclude the reliability of test results simply on basis of number of visitors.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/","og_locale":"en_US","og_type":"article","og_title":"A\/B Split Testing: Everything You Need To Know - VWO","og_description":"An essential primer on mathematics of a\/b split testing so that you never conclude the reliability of test results simply on basis of number of visitors.","og_url":"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/","og_site_name":"Blog","article_publisher":"https:\/\/www.facebook.com\/vwoofficial\/","article_published_time":"2010-01-26T16:26:47+00:00","article_modified_time":"2025-04-30T10:50:57+00:00","og_image":[{"width":1200,"height":630,"url":"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2010\/01\/OG-image_What-you-really-need-to-know-about-mathematics-of-AB-split-testing.png","type":"image\/png"}],"author":"Paras Chopra","twitter_card":"summary_large_image","twitter_creator":"@paraschopra","twitter_site":"@VWO","twitter_misc":{"Written by":"Paras Chopra","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/#article","isPartOf":{"@id":"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/"},"author":{"name":"Paras Chopra","@id":"https:\/\/vwo.com\/blog\/#\/schema\/person\/6a4fd0d62df48bca6a31f484f7c4e5b6"},"headline":"What You Really Need To Know About The Mathematics Of A\/B Split Testing","datePublished":"2010-01-26T16:26:47+00:00","dateModified":"2025-04-30T10:50:57+00:00","mainEntityOfPage":{"@id":"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/"},"wordCount":924,"publisher":{"@id":"https:\/\/vwo.com\/blog\/#organization"},"image":{"@id":"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/#primaryimage"},"thumbnailUrl":"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2010\/01\/Feature-image_What-you-really-need-to-know-about-mathematics-of-AB-split-testing.png","articleSection":["Calculator"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/","url":"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/","name":"A\/B Split Testing: Everything You Need To Know - VWO","isPartOf":{"@id":"https:\/\/vwo.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/#primaryimage"},"image":{"@id":"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/#primaryimage"},"thumbnailUrl":"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2010\/01\/Feature-image_What-you-really-need-to-know-about-mathematics-of-AB-split-testing.png","datePublished":"2010-01-26T16:26:47+00:00","dateModified":"2025-04-30T10:50:57+00:00","description":"An essential primer on mathematics of a\/b split testing so that you never conclude the reliability of test results simply on basis of number of visitors.","breadcrumb":{"@id":"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/#primaryimage","url":"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2010\/01\/Feature-image_What-you-really-need-to-know-about-mathematics-of-AB-split-testing.png","contentUrl":"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2010\/01\/Feature-image_What-you-really-need-to-know-about-mathematics-of-AB-split-testing.png","width":1200,"height":700,"caption":"What You Really Need To Know About Mathematics Of Ab Split Testing"},{"@type":"BreadcrumbList","@id":"https:\/\/vwo.com\/blog\/what-you-really-need-to-know-about-mathematics-of-ab-split-testing\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/vwo.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Calculator","item":"https:\/\/vwo.com\/blog\/calculator\/"},{"@type":"ListItem","position":3,"name":"What You Really Need To Know About The Mathematics Of A\/B Split Testing"}]},{"@type":"WebSite","@id":"https:\/\/vwo.com\/blog\/#website","url":"https:\/\/vwo.com\/blog\/","name":"Blog","description":"","publisher":{"@id":"https:\/\/vwo.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/vwo.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/vwo.com\/blog\/#organization","name":"VWO","url":"https:\/\/vwo.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/vwo.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2018\/09\/VWOLogo.png","contentUrl":"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2018\/09\/VWOLogo.png","width":780,"height":492,"caption":"VWO"},"image":{"@id":"https:\/\/vwo.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/vwoofficial\/","https:\/\/x.com\/VWO","https:\/\/www.instagram.com\/vwoofficial\/","https:\/\/www.linkedin.com\/company\/vwo"]},{"@type":"Person","@id":"https:\/\/vwo.com\/blog\/#\/schema\/person\/6a4fd0d62df48bca6a31f484f7c4e5b6","name":"Paras Chopra","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/vwo.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2019\/04\/paras-150x150.jpg","contentUrl":"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2019\/04\/paras-150x150.jpg","caption":"Paras Chopra"},"description":"I started Wingify in early 2009 to enable businesses to design and deploy great customer experiences for their websites and apps. I have a background in machine learning and am a gold medalist from Delhi College of Engineering. I have been featured twice in the Forbes 30 under 30 list - India and Asia. I'm an entrepreneur by profession and my curiosity is wide-ranging. Follow me at @paraschopra on Twitter. You can email me at paras@wingify.com","sameAs":["https:\/\/invertedpassion.com\/","https:\/\/in.linkedin.com\/in\/paraschopra","https:\/\/x.com\/paraschopra"],"url":"https:\/\/vwo.com\/blog\/author\/paraschopra\/"}]}},"_links":{"self":[{"href":"https:\/\/vwo.com\/blog\/wp-json\/wp\/v2\/posts\/60","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/vwo.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/vwo.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/vwo.com\/blog\/wp-json\/wp\/v2\/users\/14"}],"replies":[{"embeddable":true,"href":"https:\/\/vwo.com\/blog\/wp-json\/wp\/v2\/comments?post=60"}],"version-history":[{"count":33,"href":"https:\/\/vwo.com\/blog\/wp-json\/wp\/v2\/posts\/60\/revisions"}],"predecessor-version":[{"id":88169,"href":"https:\/\/vwo.com\/blog\/wp-json\/wp\/v2\/posts\/60\/revisions\/88169"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/vwo.com\/blog\/wp-json\/wp\/v2\/media\/56806"}],"wp:attachment":[{"href":"https:\/\/vwo.com\/blog\/wp-json\/wp\/v2\/media?parent=60"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/vwo.com\/blog\/wp-json\/wp\/v2\/categories?post=60"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/vwo.com\/blog\/wp-json\/wp\/v2\/tags?post=60"},{"taxonomy":"feature","embeddable":true,"href":"https:\/\/vwo.com\/blog\/wp-json\/wp\/v2\/feature?post=60"},{"taxonomy":"industry-type","embeddable":true,"href":"https:\/\/vwo.com\/blog\/wp-json\/wp\/v2\/industry-type?post=60"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/vwo.com\/blog\/wp-json\/wp\/v2\/product?post=60"},{"taxonomy":"role","embeddable":true,"href":"https:\/\/vwo.com\/blog\/wp-json\/wp\/v2\/role?post=60"},{"taxonomy":"region","embeddable":true,"href":"https:\/\/vwo.com\/blog\/wp-json\/wp\/v2\/region?post=60"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}