{"id":57629,"date":"2021-04-01T11:20:50","date_gmt":"2021-04-01T05:50:50","guid":{"rendered":"https:\/\/vwo.com\/blog\/?p=57629"},"modified":"2025-11-13T15:54:15","modified_gmt":"2025-11-13T10:24:15","slug":"predict-ab-test-results","status":"publish","type":"post","link":"https:\/\/vwo.com\/blog\/predict-ab-test-results\/","title":{"rendered":"Predict A\/B test results using ancient Indian astrology"},"content":{"rendered":"\n

We at VWO have a mission to help marketing and product teams reduce the time and effort required to figure out what works best for their business and what doesn\u2019t. In 2010, we pioneered the Do-It-Yourself Visual Editor<\/a> for business teams for editing webpages and creating their variations for A\/B tests without involving IT teams. That innovation cut the effort to launch an A\/B test from weeks to hours and the A\/B testing industry hasn\u2019t been the same ever since.<\/p>\n\n\n\n

But, as anyone who has run an A\/B test knows, you still have to wait for weeks in order to start getting statistically significant results about which version is better.<\/p>\n\n\n\n

Download Free: A\/B Testing Guide<\/span><\/a><\/em><\/h3>\n\n\n\n

We\u2019ve been wondering if there\u2019s a way to cut this wait period as well. Wouldn\u2019t it be wonderful if there was a way to instantly know which version is going to win without waiting weeks or months for the data to arrive?<\/p>\n\n\n

Can AI help?<\/h2>\n\n\n

The first research direction we turned to was the booming AI technology. You would have certainly heard about neural networks, machine learning, and data science. Could one such technique help us? If neural networks can drive cars today, perhaps they can also predict which website version is better.<\/p>\n\n\n\n

Over the last few years, we tried several techniques and invested a lot of money in trying to get one of the AI techniques to work. We did make some progress when a couple of months ago, we launched an AI-powered website copywriter<\/a> that helped marketers come up with new alternative headlines, CTAs, and product descriptions to test.<\/p>\n\n\n\n

But, unfortunately, similar AI-based techniques didn\u2019t perform satisfactorily on the prediction of A\/B test results. It seems driving cars, beating the World<\/a> Go champion, and detecting tumors<\/a> is an easier problems than predicting which website design is going to be a better choice.<\/p>\n\n\n

Enter IA<\/h2>\n\n\n

As we were about to almost give up on our dream project, someone pointed us to the elephant in the room: astrology. Our first reaction was to dismiss this absurd suggestion immediately. We\u2019re a skeptical bunch and there\u2019s no place for such pseudoscience in a serious business like ours.<\/p>\n\n\n\n

But we were desperate and decided to give this totally absurd idea one last shot. We were told that millions of people across the world predict all sorts of phenomena using techniques like horoscopes, or Tarot cards. If people are using a technique to predict whether their marriage will succeed or not, can\u2019t it be used to predict whether the A version is better than the B version?<\/p>\n\n\n\n

After looking at the cost-benefit equation, we decided to do a small pilot. If, as we predicted, astrology didn\u2019t work we\u2019ll just lose some investment and time. However, if it did work, it would change the marketing and user research industry forever.
So, in all earnestness, we kickstarted the pilot. After doing initial research on the most predictive astrology technique, we settled on Indian Astrology (IA). Particularly, we settled on parrot astrology.<\/strong> What gave us confidence in it was that there\u2019s even a Wikipedia page on it<\/a>.<\/p>\n\n\n\n

\"parrot
Photo via Vijay S (on <\/em>Flickr<\/em><\/a>)<\/em><\/figcaption><\/figure>\n\n\n\n

In parrot astrology, an astrologer lays out multiple options in front of a caged pet parrot. When the parrot is let out of the cage, it picks one of the options as the prediction. Simple and straightforward. We loved it already.<\/p>\n\n\n\n

What we did for our pilot is the following:<\/p>\n\n\n\n