Introduction to the three pillars of this blog and the three key takeaways for the modern experimenter….

The research culture in the Data Science team at VWO and the story of how we learned all that we plan to share in this blog….

A detailed summary of the structure of this blog, the broader subsections it will have and the ideas I will discuss….

An introduction to the historical context of experimentation and the deeper philosophical questions it has evolved through….

An introduction to where do modern A/B tests come from and what problem did they solve in the larger scientific context….

A discussion on the fundamental limitations of Randomised Controlled Trials and how do they apply to modern day A/B testing….

An account of the nuances of randomness in modern computing and how it matters in A/B testing….

What is the dividing line between scientific and non-scientific? An introduction to the theory of falsification and its implications on experimentation….

Fidelity in research refers to how closely the experimental setup mimics the real world. Understand the implications of fidelity in modern day A/B teting….

What are metrics and why are they needed in experimentation? An introduction to the subsection on metrics….

The three kinds of metric that serve different purposes in experimentation. Understand how modern businesses develop an arsenal of metrics to guide their experimentation efforts….

11 questions to ask when designing metrics corresponding to the desirable characteristics a metric should have. The post covers three major aspects: statistical, logistical and usability….

The larger framework of assumptions and uncertainties that statistics has been built upon and how does a statistician navigate through these problems….

An introduction to statistical significance, what does it mean and why do we need it in experimentation….

An intuitive explanation of the three forces behind statistical significance: effect size, standard deviation and sample size….

The statistical explanation behind how the different forces of statistical significance are related to each other….

An introduction to base rates – the fundamental difference between Bayesian and Frequentist schools of thought. Understand when and why base rates matter in statistical testing….

A perspective on two different kinds of probabilities that are at the crux of the difference between the Bayesian and the Frequentist school of thought….

VWO’s learnings on the fundamentals of the Bayesian vs Frequentist Debate. The differences between the two schools of thought and why does it matter….

A deep dive into the Frequentist solution to solve problems of statistical inference. The article explains the elements common to all Frequentist methods: the null hypothesis, the statistic and the p-…

A deep dive into the Bayesian solution to the problem of statistical significance. The article explains: the Prior, the Bayes Rule and the Posterior….

A discussion of the criticisms that the two statistical factions have towards each other: Frequentists, who get criticised for p-values, and the Bayesians, who find it difficult to define the priors….