You might wonder what\u2019s the deal with segmentation in the A\/B test and why it makes such a significant impact.<\/p>\n\n\n\n
To put it simply, segmentation is similar to the meticulous strategies employed by teams in the Big League, where they segment opponents’ team players based on recent form and performance and experiment with their gaming approach. No one-size-fits-all strategy here \u2014 on such a professional and super-competitive level, the teams adjust their game plan for both in-form and out-of-form segments of opponent batters and pitchers. This approach ensures that a team can apply strategies in different situations, understand players effectively, and utilize teammates’ potential.<\/p>\n\n\n\n
Similarly, you can\u2019t always have a general A\/B testing approach when you have multiple segments of visitors on your digital platform. For example, it\u2019s far more effective to A\/B test your checkout page for frequently abandoning segments than all the visitors. Segmentation gives specific insights into visitor behavior<\/a> and optimizes the bandwidth of your A\/B testing tool. <\/p>\n\n\n\n So, if you’re just dipping your toes to explore segmentation in A\/B testing, you’ll definitely want to check out this blog post. We’ll cover the how-to\u2019s of A\/B test segmentation and best practices and show you how easy segmentation is when you are using a platform like VWO.<\/p>\n\n\n\n When you’re running an A\/B test, you can carry out pre and post-segmentation. Pre-segmentation is all about digging into existing visitor data to create visitor segments<\/a> for a test. And post-segmentation? That’s breaking down the results you get from the test. Here are some commonly used filters to create segments for pre- and post-segmentation in A\/B testing:<\/p>\n\n\n\n [For instance, consider using a country filter to create segments like the US, India, UK, etc.]<\/p>\n\n\n\n Now, let’s take a closer look at how pre- and post-segmentation can be useful in your A\/B testing.<\/p>\n\n\n \u201cI am seeing a rise in cart abandonment, what should I do?\u201d<\/p>\n\n\n\n \u201cHow do I reduce that bounce rate from Japan?\u201d<\/p>\n\n\n\n \u201cHow can I improve the completion rate of that course?\u201d<\/p>\n\n\n\n These questions naturally pop up after digging into the website data from our analytics platform. Solving these issues might involve A\/B testing with some hypotheses you come up with. Well, here pre-segmentation can come in handy. Instead of testing everything on all visitors, here is how you can solve the above problems and similar ones that you face:<\/p>\n\n\n\n \u201cI will segment cart abandoners and run an A\/B test by positioning the promotion block above the fold.\u201d <\/p>\n\n\n\n \u201cLet\u2019s A\/B test Japan traffic with a translation widget in the variation.\u201d<\/p>\n\n\n\n \u201cLet\u2019s A\/B test low-engagement students segment with a variation having smaller duration videos.\u201d<\/p>\n\n\n\n With this approach of pre-segmentation, you will more likely see an increase in your conversion rate, as you are targeting segments that are a key source of the problem. Even if it fails, you can move to test the next hypothesis without subjecting a larger visitor base to the experiment. Also, pre-segmentation optimizes your use of the A\/B testing tool as many of these platforms charge based on the usage-based pricing model. <\/p>\n\n\n\n Keep in mind:<\/strong> The effectiveness of pre-segmentation hinges on the tool at your disposal and how adeptly you handle and assimilate visitor data. Contemporary businesses lean on sophisticated solutions, such as customer data platforms (CDPs)<\/a>, to store and seamlessly integrate data from diverse sources. This not only enhances the accuracy of segmentation but also boosts the results of activities driven by these segments.<\/p>\n\n\n You have finished A\/B testing, and now it\u2019s time to delve into the reports with post-segmentation. It might seem like a bit of an optional thing, but whether your A\/B test hits the jackpot or takes a detour, post-segmentation is the secret sauce\u2014it spills the beans on why things went the way they did, laying the groundwork for future improvements.<\/p>\n\n\n\n In post-segmentation, you can filter all the visitor data for a particular segment to understand how it performed or you can square off two segments like mobile versus desktop or returning visitors against the new ones, revealing the nuances in their experiences and how it impacted the conversion rate. <\/p>\n\n\n\n Moreover, delving into post-segmentation after a test flop can bring you closer to understanding customer expectations and needs. Take the example of Uncommon Knowledge, an online education portal<\/a>. They hit a roadblock in their A\/B test featuring the latest design trends. Still, the post-segmentation analysis revealed a key insight: their primary audience predominantly falls within the 45+ age group, who are not used to the latest trends.<\/p>\n\n\n\n So, when you engage in post-segmentation after an A\/B test, you’re not slamming the brakes on A\/B tests; you’re building on them. This approach addresses the common challenge of hitting a creativity wall with A\/B testing as you generate fresh hypotheses through in-depth analysis. It is a more strategic approach\u2014what we like to call the experimentation loop<\/a>\u2014to A\/B testing. In this loop, you launch a new experiment based on a hypothesis from analyzing a completed A\/B test. It’s a far superior method compared to the randomness of traditional testing.<\/p>\n\n\n\n Now, let’s dive into the practical side of things and explore how to effectively carry out segmentation using a tool like VWO.<\/p>\n\n\n Let\u2019s understand the segmentation functionality in VWO with an example. Say you\u2019ve got a cool smartwatch e-commerce store<\/a>. In recent analytics reports, you notice a decline in the conversion rate among visitors using the Windows browser. The analysis led to a hypothesis of adding a sitewide time-based discount offer widget to see whether it uplifts the conversion from Windows visitors. You create an A\/B test variation having the widget below the main menu.<\/p>\n\n\n\n With VWO, if you are thinking of running an A\/B test to see the effect of this offer on visitors with a Windows browser, you can pull it off with just a few clicks, thanks to its user-friendly dashboard. Once you’ve punched in the web URL you want to test and sort out your variations and goals, it’s a breeze to pre-segment using the standard segment option tucked away in the audience and traffic section. Here is how it\u2019s done on VWO.<\/p>\n\n\n\n Now, for the same time-based widget, if you want to kick things up a notch and dive into some advanced pre-segmentation \u2014like you have a hypothesis to A\/B test visitors who have a 60-second time spent duration on the homepage. For such use cases custom segment is your playground. You can segment users by behavior-based segments such as exit intent, time spent, and scroll-based criteria. Additionally, you can segment by location, IP address, query parameters, or anything specific to your site.<\/p>\n\n\n\n Here’s how it works:<\/p>\n\n\n\n Also, you can include, exclude, and even mash up segments for your A\/B test. For example, a combined segment of people who have 60 seconds spent on the watch store homepage from Idaho. <\/p>\n\n\n\n You can even create segments based on attributes you’ve tracked with third parties like Clearbit, Segment, etc. If you want to know more, dive into the nitty-gritty of custom segments with all the details in a knowledge-base article<\/a>.<\/p>\n\n\n\n Now, let’s say you’ve conducted the A\/B test for all users, and you’re keen on diving into post-segmentation using VWO to see how control and variation fared across different segments. Beyond the standard and custom segments we explored earlier in pre-segmentation, VWO has got you covered for comparing how the A\/B test played out for various segments of visitors. Like, you can see how conversions from the desktop segment fared against the mobile segment. You can even compare multiple-goal performances. <\/p>\n\n\n\n Also, imagine you’ve got a boatload of data, and you’re itching to dig deeper into specific segments. Like, maybe you’re curious to slice and dice based on countries for the mobile segment. In that case, VWO lets you slice and dice those segments with visitor dimensions<\/a>, making it a breeze to unravel the nitty-gritty details of the performance. <\/p>\n\n\n\n If you notice, VWO is like your trusted toolkit, handing you all the controls you need to finely segment your A\/B tests during both pre and post-stages. If you find this intriguing and want to give it a spin, we offer a 30-day full-feature<\/a> trial for a complete hands-on experience. Remember to give it a go! It\u2019s worth mentioning here that access to some of the above features depends on the chosen plan and pricing<\/a>. <\/p>\n\n\n\n Now that we’ve got the scoop on segmentation and the tool to wield, let’s dive into some savvy best practices to make the most out of it.<\/p>\n\n\n After applying basic filters like country, age, and device in your analytics tools, spot the segment with the most visitors. Make it your A-list for testing, and break it down even more based on conversion rates.<\/p>\n\n\n\n Now, for the high-converting sub-segment, A\/B testing with new features, the latest arrivals, or step up your game with website personalization<\/a>. These segments are your core visitors. If an A\/B test variation clicks with them, consider integrating it into the regular experience for all visitors.<\/p>\n\n\n\n For the low-converting sub-segment, dive into the analysis using user behavior analytics tools <\/a>like heatmaps, session recordings, and clickmaps to pinpoint friction points. Formulate hypotheses based on your findings and run A\/B tests to observe the impact on the conversion rate. It’s about diagnosing the issues, crafting solutions, and fine-tuning for better results.<\/p>\n\n\n Using the same filters as before in your analytics, you might spot small segments with a high conversion rate. This could be a sign of an untapped visitor base. For these segments, consider crafting a new experience or even developing a variation landing page dedicated to this specific segment and conduct a Split URL test<\/a>. To test the waters, increase traffic through paid and social promotion to observe if the newly found visitor base maintains a consistently high conversion rate. Such an approach to experimentation helps in exploring and capitalizing on hidden opportunities.<\/p>\n\n\n\n Say, for instance, in your analysis using a country filter, you might stumble upon a small segment of visitors from Canada (a region you don’t usually target) with surprisingly high conversion rates. In this scenario, consider crafting a landing page tailored specifically for Canadian visitors and test it against your original landing page using a Split URL test. To validate your findings, run a paid promotion to drive more traffic and ensure statistical significance<\/a> in your tests. If the results are promising and show a positive trend, you might consider implementing long-term strategies like content marketing to fully tap into this previously untapped market.<\/p>\n\n\n The allure of stopping an A\/B test midway can be strong, especially when you see a higher conversion rate for a variation, thanks to the real-time reports most tools provide. However, this impulsive decision might lead to a false positive (type-1 error)<\/a> result, and conducting post-segmentation based on this premature data could lead to inaccurate assumptions for future actions.<\/p>\n\n\n\n![\"Feature](\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2023\/12\/Feature-image-Strategic-Splits_-A-Guide-to-AB-Test-Segmentation.png\")
<\/figure>\n<\/figure>\n\n\nLet\u2019s understand how to segment A\/B tests<\/h2>\n\n\n
\n
Pre-segmentation: The slice before the spice<\/h3>\n\n\n
![\"Pre-Segmentation\"](\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2023\/12\/Pre-segmentation.jpg\")
<\/figure>\n<\/div><\/figure>\n\n\n\nPost-segmentation: Unraveling the puzzle<\/h3>\n\n\n
![\"Post-Segmentation\"](\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2023\/12\/Post-segmentation.jpg\")
<\/figure>\n<\/figure>\n\n\n\n![\"Audience](\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2023\/12\/Audience-Demographics.png\")
<\/figure>\n<\/figure>\n\n\n\nHow does VWO handle pre- and post-segmentation?<\/h2>\n\n\n
![\"Watch](\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2023\/12\/Screenshot-2023-11-27-at-3.44.29-PM.png\")
<\/figure>\n![\"Standard](\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2023\/12\/gif-seg.gif\")
<\/figure>\n![\"Custom](\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2023\/12\/gif-seg-2-2.gif\")
![\"Post-segmentation](\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2023\/12\/Screenshot-2023-11-27-at-4.59.52-PM.png\")
<\/figure>\n![\"Post-segmentation](\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2023\/12\/Screenshot-2023-12-07-at-1.14.10-PM-1.png\")
<\/figure>\nSegmentation strategies for A\/B test analysis<\/strong><\/h2>\n\n
a. Prioritize bigger segments for A\/B testing<\/h3>\n\n\n
b. A\/B test new experiences for small segments with high conversion rates<\/h3>\n\n\n
c. Dive into post-segmentation only after the test reaches statistical significance<\/h3>\n\n\n
![\"Combining](\"https:\/\/static.wingify.com\/gcp\/uploads\/sites\/3\/2023\/12\/Screenshot-2023-11-29-at-12.10.05-PM-1.png\")
<\/figure>\n