Experiment with ad copy

How to interpret the ad copy test results

Ad copy tests are experiments; the results are intended to provide you with additional data to make decisions about your creatives. The results aren't intended to guarantee an ad's performance.  If a test collects enough data to indicate that the test results aren't random, meaning that the test is significant,  then you are likely to see the same performance if you repeat the test.   

 

Examples:

Result: Is significant

If an ad copy test result is significant, and one ad performed better than the other ads,  you can stop the poor performing ads. The best ad will continue to perform within the range specified by the confidence interval.

Ad copy test results. Two ads shown, one with down arrow and one with up arrow.

Ad copy test settings

Goal = CTR
Statistical significance threshold = 95%

 

Ad copy test results for an ad

Confidence interval = 3.92% to 4.27%

The result indicates that you can be 95% certain that the true value of the CTR for the better performing ad is between 3.92% and 4.27%. 

 

 

Result: Not significant

If an ad copy test result is not significant, and all of the ads in the test perform about the same, you can stop the test and continue to run the ads.

Significant ad copy test with undetermined rating for all four ads

Ad copy test settings

Goal = Action rate
Statistical significance threshold = 90%

 

Ad copy test results 

Confidence interval for all of the ads ranges from 81.65% to 90.88%

The result indicates that there isn't a significant difference in ad performance. You can stop the test without pausing any of the ads.  If you want to test more,  consider adding more ads to the ad group and then start a new test. 

 

 

Result: Is significant

An ad copy test can be significant with all of the ads rated undecided.  

Ad copy test. Significant results, but both ads are rated undecided.

Ad copy test settings

Goal = CTR
Statistical significance threshold = 95%

 

Ad copy test results 

The confidence intervals show that Ad1's CTR is between 1.55% and 1.80%. Ad2's CTR is somewhere between 1.33% and 1.56%.  The results indicate that it's possible, but unlikely, that Ad2 could outperform Ad1. 

You have at least two options with such results. Because Ad1 performed slightly better, you can stop the test and pause Ad2. This will probably increase the CTR. Or, you can stop the test and let both ads continue to run.

 

 

Result: Is significant

An ad copy test can be significant and identify poor performing ads with undetermined ad performance.

Completed ad copy test. Results show two ads that are undecided and another that performed poorly.

 

Ad copy test settings

Goal = Revenue/visit
Statistical significance threshold = 90%

 

Ad copy test results 

The result indicates you can be 90% certain that you can stop the test and pause the poor performing ads. The other ads will continue to perform within the ranges specified.

 

Was this helpful?
How can we improve it?