Glossary of terms
In order to get the most out of Optimize, it’s important to understand a few concepts and terms.In this article:
- A/B test
- Baseline conversion rate
- Editor page
- Estimated conversion rate
- Experiment sessions
- Fractional factorial experiment
- Full factorial experiment
- Multivariate test
- Probability to beat baseline
- Probability to be best
- Redirect destination
- Redirect test
(A/B/n test, bucket test, split test)
A randomized experiment using two or more variants of the same web page (A and B). Variant A is the original and variant B through n each contain at least one element that is modified from the original. Learn more about how to create an A/B test.
Baseline conversion rate
A modeled conversion rate attributed to the variant based on the percentage of sessions that resulted in a conversion (i.e. the experiment objective being met).
In a multivariate experiment, which contains multiple sections, combinations are the number of variants in each section multiplied by each other. See the example in Multivariate test. In a multivariate test with multiple sections, a combination is the experience created from each section's variants. For example, headline 1 with hero image A.
The web page used to create variants. Note that this may differ from your URL targeting rules.
Estimated conversion rate
(Modeled conversion rate, Projected conversion rate)
Based on the data to date, this is the conversion rate you can expect to see in the future. Because Optimize takes into account the day of week and other sections into its statistical simulations, this projected rate may differ from the historical rate measured so far.
Any session that has seen an experiment variant, and all future sessions for that user for the duration of the experiment (whether they see the variant again or not). Learn more about sessions in the Analytics 360 help center.
Fractional factorial experiment
Only a subset of combinations is served in order to find results quickly. These experiments sacrifice the detailed interaction analysis available with full-factorial experiments for greater speed in finding results.
Full factorial experiment
Every combination of a multivariate experiment is served to users, to understand all interactions between variants. These generally take longer to reach significance but also allow for detailed measurement of interactions, as well as offering the ability to serve the winning combination.
For a given objective, the difference in conversion rate - measured as a percentage - between the variant and the baseline.
(MVT, compound test)
An experiment that tests two or more sections to understand their effects on each other. For example, variants of a headline can be tested at the same time as variants of a hero image. Instead of showing which page variant is most effective (as in an A/B test), a multivariate test identifies the most effective combination of variants. Rather than the two or three page variants found in simple A/B tests, multivariate tests frequently test multiple variants of multiple page elements simultaneously. Learn more about how to create a multivariate test.
The website functionality you wish to optimize.
Your current web page, prior to any modification.
Probability to beat baseline
The probability that a given variant will result in a conversion rate better than the original's conversion rate. Note that with an original and one variant, the variant's Probability to beat baseline starts at 50 percent (which is just chance).
Probability to be best
The probability that a given variant performs better than all other variants.
A separate and unique web page used as a variant in a redirect experiment.
(Split URL test)
A redirect test is a type of A/B test that allows you to test separate web pages against each other. In redirect tests variants are identified by URL or path instead of an element(s) on the page. Redirect tests are useful when you want to test two very different landing pages, or a complete redesign of a page. Learn more about how to create a redirect test.
A single element of a web page (e.g. a headline, image, or button) that is modified to create variants. An A/B test only contains one section (with one or more variants); in a multivariate experiment, multiple sections are tested at the same time.
A variant can be anything from a change to a single element, or changes to multiple elements, or a totally different page in an experiment. In an A/B experiment the unit of variant can be a web page or an element of a web page. In a multivariate experiment you have multiple variant of each section.