{"id":604,"date":"2022-03-15T07:16:42","date_gmt":"2022-03-15T07:16:42","guid":{"rendered":"https:\/\/vwo.com\/glossary\/?p=604"},"modified":"2025-02-20T07:03:43","modified_gmt":"2025-02-20T07:03:43","slug":"type-1-error","status":"publish","type":"post","link":"https:\/\/vwo.com\/glossary\/type-1-error\/","title":{"rendered":"Type-1 Error"},"content":{"rendered":"\n

What is a type-1 error?<\/strong><\/h2>\n\n\n\n

Consider a situation where an eCommerce business wants to improve its existing sales. They build a hypothesis that by optimizing the design of the product page<\/a> they could improve the checkout rate which eventually would lead to more purchases. <\/p>\n\n\n\n

To validate this hypothesis, they A\/B tested the two designs<\/a> and measured the click rate on the checkout button. According to the hypothesis testing approach of an A\/B test, after running the experiment for a certain duration, if it declares a statistically significant result in favor of the new product page design then the new hypothesis is considered to be true.<\/p>\n\n\n\n

Now suppose, the test results were wrong and truly there wasn\u2019t any difference between the two designs, then the test has committed a False positive or Type-1 error. If an A\/B or a Multivariate test<\/a> declares a statistically significant result when in reality no difference exists in the performance of the variations being tested, then it is a Type-1 error.   <\/p>\n\n\n\n

In scientific terms, during the hypothesis testing process when a null hypothesis (representing no effect) is rejected, even if it is accurate and should not be rejected by the test then it is called a Type-1 error or a False Positive. A null hypothesis is defined before the start of an A\/B test or MVT which represents no difference between the variations being tested. <\/p>\n\n\n\n

To put it more formally, in an A\/B test, if both variations are similar and don\u2019t affect the metric being tested any differently, an error may occur where the null hypothesis is rejected after the test concludes. In such a case, if it’s determined that there is a statistical difference between the variations, the result is a Type I error.<\/p>\n\n\n\n

Why is it important to understand type-1 errors?<\/strong><\/h2>\n\n\n\n

After running an A\/B test suppose you incorrectly conclude that variation B is a winner and deploy it for all traffic. A wrong conclusion can be detrimental to the conversion rate of a business and can lead to revenue loss. So whenever you run an A\/B test, an understanding of type 1 errors<\/a> can help you to  <\/p>\n\n\n\n