Use A/B tests to make informed decisions about how to set up your ad sources in mediation, which can help you increase your earnings.
On this page
How A/B tests work
A/B tests let you evaluate 2 variations of a mediation group and compare the results. Use an existing mediation group and create a variation of it to see which performs better. The test splits your app’s users between your existing mediation group setup and the variation, so that their performance can be measured side by side.
You can change your mediation group set up in Variant B by:
- Adding or removing ad sources to see their impact on your mediation group
- Using a different waterfall order by changing the manual eCPM value of ad sources
- Enabling or disabling optimization on waterfall ad sources
- Changing the number of times an ad source is called
Click to view an example test
The test runs for 90 days, but you can stop at any time. Based on the results of the test, you can decide to use the original mediation group setup (Variant A) or the variation (Variant B).
Set up an A/B test
Complete the following steps:
- Sign in to your AdMob account at https://admob.google.com.
- Click Mediation in the sidebar.
- Hover over the mediation group that you want to create an A/B test for and click
. Then, click Set up A/B test.
- Enter a name for your test.
- Use the slider to set the user allocation. This determines the percentage of app users that are allocated to variant B during the A/B test. You can assign from 1% up to 50% of your app user traffic to variant B. The remainder will remain in variant A (original).
Note: Once you set the users' allocation and begin the test, you can’t change it.
- If you’ve already conducted an A/B test for a mediation group, you can use the Variant B dropdown to copy and rerun the variant B settings from a previous test.
- Change the ad source configuration in Variant B. You can:
- Add or remove ad sources to see their impact on your mediation group
- Set the bidding eCPM floor to A/B test different floor values
- Change the order of your waterfall by changing the manual eCPM value
- Enable or disable optimization on waterfall ad sources
- Change the number of times an ad source is called
- To ensure accurate reporting, we recommend using unique ad unit mappings when using the same ad source in Variant A and B. For example, use a mapping that isn't already used in variant A.
- Click Edit in the Ad unit mapping column. In most cases, the mapping details are available in your account with the ad network.
- Learn more about adding waterfall ad sources and bidding ad sources.
- Click Save.
- Click Start test.
Once you start the A/B test, you won’t be able to make changes to the mediation group, including Variant A or Variant B, until the test is stopped or has completed. Any change while the experiment is running may impact the results. This means you can’t edit ad units, location targeting, etc.
- Click Start in the confirmation box, and the test will begin in a few minutes.
. Then, click Set up A/B test. By default, the test runs for 90 days but you can choose to take action and end the test based on the data available. Learn how to analyze your A/B test and take action.
Best practices
- Choose ad units that have enough traffic for the test to determine a result. A test must receive at least 10,000 ad requests before AdMob can determine a result.
- Change only one setting at a time so you can more accurately attribute any performance difference to a particular change.
- Run the test for at least two weeks. More data helps us more confidently estimate which variant will perform best.
- Consider using different mappings in Variant A and B. This helps keep test data and reporting accurate for your waterfall ad sources.