SmartStats is designed to give you intelligent answers quickly to make smart business decisions. Compared to traditional frequentist-based tools, SmartStats usually provides answers in less time unless your base conversion rate is above 15%.
It’s important to note that we gain this speedup through several trade-offs than the traditional frequentist methods. Most traditional statistical methods were designed for publishing scientific papers and to determine whether an effect is real. The stakes are different on the web. Either variation A or variation B is deployed; the question a company asks is “Which option will make us more money?”
You should run your test for an integer number of weeks, and the VWO app warns you if you stop the test after five or eight days. As the test (which contains one Monday for every Saturday) is supposed to be representative of the future, it’s usually a best practice to make sure that the test has the same number of Mondays and Saturdays in the test period.