Note: We are rolling out Mutually Exclusive Campaign Groups feature for Testing (A/B, MVT, Split URL) and Personalization campaigns in gradual phases. If you want an early access to the feature, please reach out to our Support team.
Many VWO users set up multiple tests on the same page and then want to run these simultaneously. In such situations, a visitor on the page can become a part of multiple tests running on the same URL, and the visitor data is collected from all the test reports.
For unrelated tests, it does not matter if your visitors become a part of more than one test. However, if the tests are related, you may want to keep the visitors exclusive to each test and avoid visitor overlap and interaction effects like in the following cases:
- Running multiple tests on the same page
- Running multiple tests on the same funnel where there is a possibility of visitor overlap (like checkout or form pages)
You can create mutually exclusive groups to make your tests exclusive, and any visitor who is a part of one test does not become a part of another. By ensuring that your visitors are not exposed to multiple tests running on the same page, you can keep the reports clean, attribute the change in conversion rate to the correct test, remove bias, and avoid overlapping visitors across tests. Please note that a test can only become a part of one exclusive group.
Let us say you want to run a test by changing the color and position of your primary CTA on the home page to make it more prominent on the page. Simultaneously, you also want to run another test on the same page by changing the headline of the page to test if an idea is communicated effectively to the users.
In such scenarios, if you run both tests at the same time and track the increase in engagement on the page, you cannot be sure which change delivered positive or negative results. In other words, you can not say accurately which set of change had the maximum impact. Creating mutually exclusive groups allows you to run these tests without any overlapping visitor data between the tests.
Using the Mutually Exclusive Groups option, you can group multiple tests together to make sure that visitors who become part of one test do not become part of another test in the same group. This means that the data collected for one test is not affected by any other test, even if both the tests are set up based on the same URLs and targeted to exactly the same set of visitors. Tests included in one group are mutually exclusive, which means that your website visitors will only see one test from a group.
Create a Mutually Exclusive Group from Campaign Settings
To create a mutually exclusive group at the account level:
- Click Settings on the right side of the VWO dashboard, and then select Campaign Settings.
- Under the Mutually Exclusive Groups section, click Create mutually exclusive group.
- Enter the name for the group, and then select campaigns you want to include in the group. You can add a maximum of 10 campaigns in a group. You must have a minimum of two campaigns in a group.
- Click Create Campaign Group.
To add your test to a mutually exclusive group:
- In the last step of the test creation process, under Advanced Options, select Add to mutually exclusive group.
- Select an existing group where you want to add the test. You can create another group by clicking Create New Mutually Exclusive group.
- To add the test to the exclusive group, click Finish.
Frequently Asked Questions
How many Mutually Exclusive Groups can I create in an account?
You can create a maximum of 10 groups in an account.
How many tests can I add in an exclusive group?
You can add up to 10 tests in a single exclusive group.
What happens if I add a running test to a mutually exclusive group? Or remove a test from an exclusive group?
This will be no different in the way the test is running, and no additional settings are required to add or remove tests from an exclusive group. From the time you add a test to a group, all new visitors will become exclusive to each test. Visitors who have already become part of the test will continue to see the previously served variations and changes.
When you remove a running test from an exclusive group, the test will continue without any disturbance. However, visitors will no longer be exclusive to the test.
To completely avoid overlapping visitors between tests, it is recommended that you add your test to the relevant mutually exclusive group first and start those tests after that.
Who can create and manage mutually exclusive groups in VWO?
Only users with Owner, Admin, or Publish-level access can create and manage the groups.
Can visitors who become part of a test in one group be considered for another test in a different group?
Yes, visitors who are part of one test in a group can become part of another group.
How do Mutually Exclusive Groups work internally?
As soon as a visitor lands on the test URL, VWO checks how many tests are running on the page and if the visitor is a part of any test in a mutually exclusive group. If not, VWO identifies the number of tests that visitor can be a part of and buckets the visitor in one campaign.
Can I add a Personalization campaign along with an A/B test in one group?
Yes, you can add A/B, MVT, Split, and Personalization campaigns to a single group.
What happens when I add a campaign with Behavior targeting to a group?
For Behavior targeting, VWO waits for an action from the visitor on the website (such as clicking a button or spending more than 30 seconds on the page) before the test runs. In such cases, we are not sure if the action will be performed when the page loads; therefore, we assume it to be true, check rest of the conditions, and then decide which test will be shown to the visitor.
If VWO decides that a campaign with Behavior should be shown to the visitor in above scenario, it will wait until the visitor performs the specified action and then include the visitor in the campaign. If selected visitors do not perform the specified action, they will not see this campaign and will be re-evaluated for the group on subsequent pages.
If I add three tests running on the same set of pages to a group, how are visitors divided between multiple tests?
When a visitor lands on a webpage with mutually exclusive tests, VWO evaluates how many tests can the visitor be a part of and then bucket the visitor in one out of all eligible tests. If, for example, a visitor qualifies for all three tests, they will have an equal chance of becoming a part of any one test. In other words, if 100 visitors land on test pages and they all are eligible for all 3 tests, they will be divided across the tests in 1:1:1 ratio.
If one campaign in a group has 50% traffic allocated to it, how does it affect the group traffic distribution?
For every visitor, VWO first evaluates the campaigns the visitors become a part of and then decides which campaign should the visitors see. During this evaluation, VWO checks all parameters including campaign URLs, segment conditions, and campaign traffic. Only when a visitor qualifies for all these parameters will the visitor become part of a campaign.