Using smart segmentation in experiment mode
Experiment mode allows you to test smart segmentation on a random 10% of your app’s users for 90 days.
For users in the 10% experiment group, ad requests for all predicted non-purchasers will be filled, while all ad requests for predicted purchasers will not be filled.
If the results of the experiment are positive or neutral after 90 days, the ad unit will automatically switch to full mode, which expands exposure of the ad unit from 10% to 100% of your users. If any of the experiment metrics are negative after 90 days, the ad unit will be automatically removed.
You may choose to end the experiment at any time. At the bottom of the experiment card, there are two options:
- Switch to full mode: Expand exposure of the ad unit from 10% to 100% of your users.
- Stop experiment: Remove the ad unit and stop all exposure.
Experiment mode reporting
You can monitor the results in the smart segmentation card on your App dashboard. The results are determined by comparing the users in the 10% experiment group that are exposed to the experimental ad unit with a control group that is equally sized and are not exposed to the experimental ad unit.
You’ll be able to measure the progress of the experiment group with the following metrics:
- Est. ad earnings: An estimated amount of revenue that the ad unit has generated during the experiment.
- Daily engagement: The percent difference between users in the experiment group and the control group for the average estimated daily session length.
- Ads ARPDAU: The percent difference between users in the experiment group and the control group for the average AdMob Network ads revenue per daily active user (Ads ARPDAU). Note that this is only for revenue generated by AdMob Network ads and doesn’t include revenue generated by in-app purchases or through mediated networks.
- 7 day retention: The absolute percent difference between users in the experiment group and the control group for the average 7 day retention rate.
- Model accuracy: The percent of purchasers who were accurately predicted by smart segmentation.
Data will only display in the metrics when the numbers are statistically significant, which means you might see:
- No data: Some or all metrics may remain empty if the experiment did not generate enough data. This could be due to insufficient traffic or incorrect implementation. Also, some metrics may be disabled if your app is not linked to Firebase.
- Neutral: The experiment hasn’t caused a statistically significant change to this metric. Statistical significance can be driven by several factors, including sample size and experiment duration.
Full mode enables smart segmentation to predict the behavior for 100% of users in an app for the ad unit. This means that ad requests for all of predicted non-purchasers will attempt to be filled for the ad unit, while all ad requests for predicted purchasers will not be filled for the ad unit.
You can use full mode as soon as you create the ad unit, or you can switch to full mode at any point during experiment mode. Switching to full mode ends the experiment and expands exposure of the ad unit to 100% of your users.
Use your AdMob reports to monitor revenue changes after enabling smart segmentation. View the ad unit with smart segmentation enabled in your AdMob Network report.