Create an A/B test
Where to begin
Before you can start testing, you need something to test on. Rather than randomly testing items on your homepage or adapting your checkout flow, start small. Test the text on an important call-to-action (CTA), change the color of a button you want users to click, or try removing extraneous fields from a form. Get more idea starters from the Example experiments article.
Create a hypothesis
Next, you need a hypothesis. A hypothesis is simply an educated guess that you'll validate or invalidate with experimentation. Think of it as the effect that you think a change will have.
"If I change the color of the 'buy now' button, revenue will increase by x."
Create an A/B test
In this experiment, we’ll test changing the color of a button:
- Go to your Optimize Account (Main menu > Accounts).
- Click on your Container name to get to the Experiments page.
- Click CREATE EXPERIMENT.
- Enter an Experiment name (up to 255 characters).
- Enter an Editor page URL (the web page you'd like to test).
- Click A/B test.
- Click CREATE.
Use a fully resolved URL in the Editor Page field. Redirects aren't supported here.
The Objectives tab
Configure your experiment objectives on the Objectives tab:
- Select a Google Analytics view.
- Select a Primary Objective.
- (Optional) Click + ADD AN OBJECTIVE to add an objective. Optimize users can use up to three pre-selected objectives per experiment and see data for those objectives in Optimize reports. Optimize 360 customers can select additional objectives and view additional goals.
- Add a description and hypothesis.
- Click SAVE.
The Targeting tab
Configure who you wish to target on the Targeting tab:
Who to target
The Who section of the Targeting tab is used to select the visitors to whom you wish to target your variant. Enter a numeric value (in tenths of a percent) or use the slider to specify the percentage of your visitors to include in your experiment.
Edit variant weights
All variants are weighted equally by default in Optimize. A visitor who is included in your experiment has an equal chance of seeing any of your variants. If you want more (or less) traffic to go a specific variant, you can adjust the Weighting of visitors to target.
To edit the weights of your variants:
- Click EDIT, next to the weight distribution bar.
- Enter values for each variant as a percentage (up to one decimal place).
- Click DONE.
Click RESET TO EQUAL to rebalance the weighting equally among your variants.
All of your variant weights must total 100 percent. If they don’t, you’ll see a red warning message. Simply adjust the weights so that they total 100 percent, or click FIX IT, which will adjust your current values to total 100 percent.
When to target
The When section of the Targeting tab determines where the experiment is shown. When is evaluated each time a user visits the experiment page. Use the rules to set where your experiment is seen. To create a targeting rule, click AND, then select a targeting rule type.
You must create either a URL or path targeting rule to start your experiment. The quickest way to get started is to create a URL matching rule with the same Editor page URL that you used when creating your experiment.
Optimize includes the following targeting rule types, which you can read more about in the following articles:
- URL targeting
- Google Analytics Audiences targeting
- Behavior targeting
- Geo targeting
- Technology targeting
- First-party cookie targeting
- Query parameter targeting
- Data layer variable targeting
Read more about Optimize targeting.
Create a variant
After being directed to the URL you entered, you'll see the Optimize Visual editor which consists of two components: the App bar at the top of the page and the Editor panel in the lower right.
- Click on the page element that you wish to alter (e.g. a button).
- Use the editor panel to change the color.
- Click Save to save the variant.
- Click Done.
Read Using the Visual editor for more information.
Start your experiment
Click START EXPERIMENT. When the Status field says Running, your first experiment is running live on the web. (Most updates happen within a minute.)
How long should your experiment run?
Keep an experiment running until at least one of these conditions has been met:
- Two weeks have passed, to account for cyclical variations in web traffic during the week.
- At least one variant has a 95 percent probability to beat baseline.
To monitor a running experiment or see the results of a concluded experiment, click the Reporting tab at the top of the experiment detail page. The report is broken down into a series of cards that contain data about your experiment, including its status and how your variants perform against your objectives.
In addition to the reports included in Optimize, you can also see Optimize reports in Google Analytics. Sign in to Google Analytics, select the Reporting tab and select Behavior > Experiments in the report navigation. Learn more about Optimize reports.