By default, campaign experiments assign a user to a test or control group every time they enter a search (this is known as 'search-based splits'). This means that if someone performs three searches, two of those searches could be subject to test treatments and the third could be subject to the control treatment. With our new cookie-based splits, you now have the option to assign searchers to dedicated test and control groups. This means that if someone is mapped to a test group and performs three searches, all three will be subject to test treatments.
Which split option you decide to use is at your discretion. It's important to understand the options and align them with what you want to test.
Cookie-based splits might be a better option in the following cases:
- Testing specific audiences
- You want to set up a landing page test that runs similarly to other conversion rate optimisation testing tools
Search-based splits might be a better option in the following cases:
- You want to keep consistency with previous search-based tests
- Low traffic volume campaigns
Let’s say that you want to test the impact of Accelerated Mobile Pages (AMP), pages which often load in under a second. If you set this experiment up using search-based splits, a given user might experience both AMP and non-AMP experiences, as their inclusion in test or control is re-assessed for each search that they do. Enabling cookie-based splits allows you to keep user experiences consistent, as the user is permanently assigned to your experiment's test or control group.
Learn more about experiments here
