App uplift experiment best practices

This article discusses best practices for app uplift experiments.

Before creating a new experiment

Understand how app uplift experiments solve your use cases

What is an app uplift experiment? App uplift experiments let you experiment and understand the performance uplift from adding video assets to your existing campaign.

We recommend the following approaches when using app uplift experiments, depending on what you want to do:
- Try video for the first time: If you don’t currently have video in your campaign, an app uplift experiment can help you understand the performance uplift from adding video assets.
- Pick the winner among multiple video assets with directional results: If you have multiple video assets, an app uplift experiment can help you understand:
  - If all the video assets collectively help you improve performance
  - How each video asset contributes to the overall performance uplift

Minimum budget and bid

We recommend a budget and bid that enable the campaign to get at least 100 (ideally, 150+) conversions per day to ensure that our models can optimise your campaigns. Smart Bidding simulators help you to better understand how many conversions you are likely to get when you change your budget and bid strategy target.

The higher the daily number of conversions in the experiment, the faster the experiment will reach statistically significant results.
If your base campaign contains a high number of existing video assets (> ~50), the budget required to assess each asset daily is likely to be much higher.

Campaign bid strategy target (tCPI/tCPE/tROAS)

If your campaign is budget-constrained, ensure that your actual CPI or CPE is not more than 2 times lower than your target CPI or CPE (and similarly for tROAS). This will help ensure that we don’t experience unexpected behaviour with cold start/bid lowering.

In general, campaigns that aren’t constrained by budget or bids will achieve quicker and more accurate results.

Check existing video assets

If the campaign is budget-constrained

If your current campaign doesn’t have any videos or has videos but isn’t spending, testing the addition of new videos is unlikely to bring a performance uplift.
Consider increasing the budget of the campaign until it’s no longer constrained and then evaluate the need for an uplift experiment.

If the campaign isn’t budget-constrained

If your current campaign has video assets but they account for a low percentage of the campaign’s total spending, testing the addition of new video assets is unlikely to bring a performance uplift.
Consider raising your target cost per conversion (or decreasing your tROAS) until your existing video assets reach a meaningful amount of spend, then evaluate the need for an uplift experiment.

Setting up an experiment

Experiment goals

It's preferable to select experiment metrics in line with your campaign optimisation goals.
- For example, choose install volume or CPI if your campaign is optimising for installs.
Choose cost per action (install/in-app action) over conversion volume metrics unless your campaign isn’t constrained by budget.

Experiment split

We recommend using a 50/50 traffic and budget split in most situations to get to the fastest experiment results possible with lowest cost.
In certain situations – such as when you think the assets you are testing will generate a large negative impact – it may make sense to use a different traffic split (for example, 40% in the trial campaign, 60% in the base campaign).

Confidence level

We recommend using an 80% confidence level as this generally provides good accuracy in experiment results with shorter duration and less cost compared to an 85% or 95% confidence level.
If you’re unsure about the confidence level to pick for your experiment, you can use the table in the appendix to find the number of conversions that you would need to reach a given confidence level.

Experiment dates

We recommend running experiments for 30 days if you can to maximise the possibility of conclusive experiment results.

Experiment Health Check

Health Check provides a series of diagnostics and checks to improve the probability of conclusive experiment results. We recommend that you fix severe issues (shown in red), such as using an iOS app (currently not supported), and try to fix moderate issues (shown in yellow) as best you can, such as budget constraints. Learn more about Health Check when creating an app uplift experiment.

General recommendations

Interactions with other campaigns promoting the same app

Ensure that the account doesn’t have another campaign promoting the same app in the same geographic locations as the campaign being tested to avoid campaign cannibalisation.

Policy violations

Fix eventual policy violations that you might have in your campaign (when possible) as these could prevent one of the campaigns in your experiment from running or could delay the results.

While the experiment is running

Budget and performance target changes

We recommend not updating these settings for the first 7 days of the experiment.
If changes are required after that, small daily incremental changes are preferable over a large change all at once.

Asset changes

If you need to make a change to an asset in your base campaign, make sure that you duplicate the change in the corresponding treatment campaign at the same time.

Monitoring experiments

We recommend excluding the first 5–10 days of the experiment from the results by using the date selector to avoid having the campaign learning period influence the metrics.
You have the option to monitor experiment results using the 3 confidence levels (80%, 85%, 95%).
If you added multiple video assets in the trial campaign, you can view the performance of an individual video asset in Google Ads reporting.

When the experiment ends

Interpreting experiment results

Statistically significant results
- Positive results for both experiment goals: We recommend that you promote the asset to your base campaign and potentially other campaigns in your account where applicable (for example, campaigns with similar goals but in a different geographical location) to improve your overall performance.
- Negative results for both experiment goals: We recommend that you do not promote the asset to your campaign or account.
- Mix of positive and negative results for experiment goals: We recommend that you decide what to do based on your business needs and ROI constraints. For example, if CPI increases by 5%, installs increase by 10%, advertisers should promote the assets if they are comfortable with more installs at a somewhat higher average CPI.
Non-statistically significant results
- We recommend that you decide what to do based on your business needs and risk tolerance. For example, for advertisers comfortable with directional results, promoting assets with positive but non-statistically significant results is reasonable. Alternatively, we recommend making changes to the asset and conducting another experiment.

Was this helpful?

How can we improve it?