{"id":53585,"date":"2020-08-21T20:39:28","date_gmt":"2020-08-21T15:09:28","guid":{"rendered":"https:\/\/vwo.com\/blog\/?p=53585"},"modified":"2025-05-05T15:34:09","modified_gmt":"2025-05-05T10:04:09","slug":"what-goes-into-an-ab-test","status":"publish","type":"post","link":"https:\/\/vwo.com\/blog\/what-goes-into-an-ab-test\/","title":{"rendered":"What Goes Into An A\/B Test And How To Improve Testing Efficiency"},"content":{"rendered":"\n

Your peer comes over to you with a test idea. They want it placed in the roadmap and prioritized accordingly. \u2018No problem\u2019, you reply. You discuss the concept, translate it into a rough wireframe on the spot, and get their approval on the idea. Your peer says, \u2018Great. So what are we thinking, can we launch this tomorrow?\u2019<\/span><\/p>\n

How often have conversations like this taken place where someone doesn\u2019t fully understand CRO, and what goes into running an experiment<\/a>? In my experience, it happens a lot more often than you think. Even those who have a crystal clear process don\u2019t necessarily know how BEST to build and run an A\/B test.<\/a> Sure, physically launching a test is as easy as clicking a big blue button, which says \u2018Launch Test,\u2019 but there is a lot more that <\/span>should go behind the scenes of building the test.<\/span><\/p>\n

Download Free: A\/B Testing Guide<\/em><\/span><\/a><\/h2>\n\n\n
\n
\"various<\/figure>\n<\/div>\n\n

Data capture and analysis<\/h2>\n\n\n

In this article, I\u2019m going to go in-depth to explain exactly what<\/strong> is involved in a single A\/B test, and what can be done to streamline the process of test building.<\/p>\n\n\n\n

The \u2018test\u2019 actually starts before any ideas even come to the table. Why? Because your ideas should be grounded in some form of data. <\/span><\/p>\n\n\n

\n
\"various<\/figure>\n<\/div>\n\n\n

Why is this important in an A\/B test? When you’re building priorities for  A\/B tests<\/a>, those with some kind of data should always be prioritized over those that do not. All tests I\u2019ve won have had some version of data feeding into it, and to date, I have not won a test that went off simply on a \u2018gut feel\u2019 or instinct. Your tests\u2019 win rate will be far higher if you run tests grounded in some form of data to influence test build. <\/span><\/p>\n\n\n\n

In each job where I started as a CRO professional, I rarely ran a test in the first month. It took me a few weeks to immerse myself into all available data, and combine that data with my observations to identify tangible things to test. <\/span><\/p>\n\n\n\n

One example of this was when I was looking at a site from the outside in, I had a number of ideas for things which I thought made sense to test. After looking at the data, I realized some of my ideas, while valid, would not be as impactful as testing into other concepts. Notably, one site I was testing into had an absurd amount of click action on a specific element that I would\u2019ve never thought would happen. Once I realized this was how users were interacting with our site, I heavily prioritized a test to capture some of the attention paid on this particular element to shift attention away from that element. <\/span><\/p>\n\n\n\n

Without spending the time digging deeper into the data, I would\u2019ve missed this obvious prioritization of a test. And it\u2019s not always easy spending time in data collection. Data collection can be quite tedious and time-consuming, and sometimes you may not have tangible next steps as a result of your collection. But try to identify ways to focus your data collection based on specific tests, concepts, or problems you\u2019re trying to solve. <\/span><\/p>\n\n\n\n

One way is to utilize help from other teams, such as UX and analytics<\/a>. Sometimes, you can identify test ideas<\/a> simply by looking at analytics and heatmaps<\/a>. But your UX team can help you further in your data collection. Talk to them about the data you have and the problem you\u2019re trying to solve. The UX team can help by providing different test executions or conducting user surveys to help you get closer to a more data-driven test.<\/span><\/p>\n\n\n\n

Here is an incredible webinar to dive deeper into the topic of hypothesis generation for a test.<\/span><\/p>\n\n\n\n

\n