VWO Logo
Dashboard

VWO SmartStats for Smarter Business Decisions

VWO’s Bayesian-powered statistics engine is designed to do the heavy lifting when it comes to calculations and accuracy for you and gives you all the ingredients you need to make the right business decisions.

Know more
Complies with:
VWO GDPR Ready Badge
VWO CCPA Ready Badge
VWO G2Crowd Leader Spring Badge
VWO BS 10012 Certification Badge

We tweet useful stuff daily

Related content:

Future is Probabilistic

3 Min Read

Hi 👋  I am Paras Chopra, founder & chairman of VWO. This is my 2nd letter in the fortnightly series with a new idea or a story on experimentation and growth.

“All models are wrong, but some are useful”.

What did the British statistician George Box mean when he wrote these now-famous words in 1976, and how is it relevant to you?

Whenever we try understanding the world around us – be it our customers’ behavior, or how stars circle the center of a galaxy, or how coronavirus affects the human body – we never have direct and full access to the underlying reality. Instead, what we have access to is only the data generated by the process that we want to understand.

So, for example, we know that a customer clicked on one button but ignored the other one. Based on this data point, we now infer what sort of customer she must be and predict her needs are and how we can fulfill them.

Notice that the data hasn’t told you a lot – it’s sterile, a mere bunch of numbers. But when you combine the fact that a customer clicked on the button with your assumptions about how people behave, you get the magical ability to predict the future (which you can use to your advantage).

But your assumptions are not equal to reality and that’s why Box said: “all models are wrong”. To understand this clearly, look at the following chart:

Future Is Probabilistic Graph

Imagine you start observing the data from the start (the bottom left) and you move in time and collect more data points (up to the blue circle). At this stage, you have to decide whether future data points will lie on the red curve or the black curve. Perhaps you have to make a business decision based on this choice. Which one will you pick?

Actually, in this case, even though black and red curves are generated through two very different realities (see their equations), there’s no way any amount of historical data can help you choose because all data up to blue circle is consistent with both.

As you can see data cannot help you select between hypotheses, it can only help you eliminate. The idea that theories can never be proved true, but only be shown to be wrong is core to how science is done. So scientists definitely know which theories are wrong, but they never know for certain which theories are right. (Surprisingly, that’s also how venture capitalists work: while investing, they know for sure which companies are “duds” but they never know which ones are going to be “unicorns”).

Box also said: “some (models) are useful”. Notice that he didn’t say some models are correct. He used the term “useful”.

The usefulness of models points to their ability to predict the future. Scientific laws (like Newton’s law of gravitation) are models that help us predict solar eclipses hundreds of years ahead. They work quite well for this purpose while another model like throwing darts to predict eclipses will fail horribly. So even though both models are wrong, Newton’s law gives us more mileage because it’s proven to be useful in a variety of contexts.

The key points to remember:

  • Don’t shoot for being right because there’s no such thing as the “correct” assumptions. Shoot for having “useful” assumptions.
  • Data alone is sterile. Whenever you think data is giving you insights, it’s actually the data+your assumptions that are informing you.

If you enjoyed reading my letter, do send me a note with your thoughts at paras@vwo.com. I read and reply to all emails 🙂

Paras Chopra
Paras Chopra I started Wingify in early 2009 to enable businesses to design and deploy great customer experiences for their websites and apps. I have a background in machine learning and am a gold medalist from Delhi College of Engineering. I have been featured twice in the Forbes 30 under 30 list - India and Asia. I'm an entrepreneur by profession and my curiosity is wide-ranging. Follow me at @paraschopra on Twitter. You can email me at paras@wingify.com
More from VWO
How Optimizing Your Checkout Page Is Different From Optimizing Your Landing Page

How Optimizing Your Checkout Page Is Different From Optimizing Your Landing Page

Your checkout page is undeniably the “holy grail” of your website if that’s where you…

Read More
Vlad Bisceanu

Vlad Bisceanu

12 Min Read
8 Tips for Optimizing Your eCommerce Homepage for Success

8 Tips for Optimizing Your eCommerce Homepage for Success

In 2020, the pandemic forced consumers to shift from making offline purchases to making online…

Read More
Leigh-Anne Truitt

Leigh-Anne Truitt

15+ Min Read
Google’s Page Experience Update 2021: Prioritize UX Optimization for Success

Google’s Page Experience Update 2021: Prioritize UX Optimization for Success

Your website represents the character of your brand in the digital world. It is the…

Read More
Niti Sharma

Niti Sharma

7 Min Read

Scale your A/B testing and experimentation with VWO.

Start Free Trial Request Demo
Invalid Email
Shanaz Khan from VWO

Hi, I am Shanaz from the VWO Research Desk.

Join our community of 10,000+ Marketing, Product & UX Folks today & never miss the latest from the world of experience optimization.

A value for this field is required.

Thank you!

Check your inbox for the confirmation mail

All you Need to Know About Conversion Optimization Get the Guide