SmartStats reduces your testing time by half

Get more intelligent results to make smarter decisions

SmartStats is our all-new Bayesian powered statistics engine for A/B testing which produces results much faster than before and gives you more control over your testing. You can now plan better, have a more accurate reason to end tests and really understand how close or far apart A and B are.

Isn’t that what you really want from your optimization platform?

SmartStats is faster than Frequentist!

Compare the average duration of a campaign on a Frequentist A/B testing tool vs Bayesian based SmartStats

Daily average number of visitors

+ -

Current Conversion Rate

+ -
%

Expected Change in Conversion Rate

+ -
%

You will get results in:

Traditional A/B testing

Using Frequentist Statistics

New SmartStats

Using Bayesian Statistics

* This is an approximate duration for an A/B test with control and one variation.
You are recommended to run a test for at least 7 days or an integer number of weeks to include weekend traffic and equal number of Mondays, Saturdays etc.
VWO’s SmartStats is so much better than the traditional statistic models most tools have used. It answers the questions a business is actually asking and helps you avoid some risk of making bad decisions that could cost you real money. It uses advanced Bayesian statistics to speed up your tests and you can put more trust in your results. Use SmartStats to make better decisions from your experiments!

Chris Goward

Founder

Get more intelligent results for making smarter decisions

Easily understand your results with recommended actions

Smart decisions minimize potential loss

Potential loss is the lift you can lose if you deployed A as the winner when in fact B was actually better. SmartStats uses potential loss to end tests. When the potential loss of a variation drops into your comfort threshold, SmartStats suggests that it’s a smart decision to deploy the variation. With a smart decision, you stand to increase your conversion rate and will, most likely, not lose anything since the potential loss from a false positive is minimized.

How smartstats uses potential loss to declare smart decisions

  • DAY 0
  • DAY 2
  • DAY 7
  • DAY 14
All tests start without any assumptions about the conversion rate (CVR) of A and B.
Instead of trying to find the absolute truth, SmartStats looks for the better choice. You get intelligent answers quickly so that you get more time to increase your conversions and more time to test more ideas.

Chris Stucchio

Ph.D, Director of Data Science - VWO

Get greater control over your testing

Depending on how much of a direct impact a test can have on your revenue, you can pick one of the three modes to help you find answers as quickly as possible.  SmartStats will end the test when the potential loss falls into your comfort zone.

  • DAY 0
  • DAY 2
  • DAY 7
  • DAY 14

As SmartStats becomes surer about the conversion rates of A and B, it better estimates the potential loss of chosing A. By choosing a mode you define a threshold and once the potential loss falls under this threshold, SmartStats ends the test and declares a smart decision.

How to choose a suitable mode

Quick Learnings

Great for finding quick trends.

When can you use it?

With this mode you can reduce your testing time for non-critical tests where there isn’t a risk of hurting your revenue directly by deploying a false winner.

Balanced

Ideal for most tests

When can you use it?

This is the default mode and can be used for almost all tests. As the name suggests it is the best balance between testing time and minimizing potential loss.

Maximum Certainty

Best for revenue-critical tests when you want to absolutely minimize potential loss.

When can you use it?

Suppose you have an e-Commerce website and you want to test changes to your checkout flow. You want to be as certain as possible to minimize potential loss from deploying a false winner even if it takes a lot of time. This is the best mode for such critical tests which affect your revenue directly.

Use a better methodology with no extra effort

The ultimate guide to SmartStats

Get everything you need to know about SmartStats and Bayesian A/B testing on email.

Who’s using SmartStats right now

Start testing the smarter way!

No credit card required

Check Your Email!
We've sent a message to
yourmail@domain.com
with an activation link in it.
Just click the link, and we'll take it from there.
Can't find the mail?
Check your spam, junk or secondary inboxes.
Still can't find it? Let us know at support@vwo.com