Glossary of terms

The language, lexicon and lingo of testing.

In order to get the most out of Optimize, it’s important to understand a few concepts and terms.

In this article:

A/B test

(A/B/n test, bucket test, split test)

A randomized experiment using two or more variants of the same web page (A and B). Variant A is the original and variant B through n each contain at least one element that is modified from the original. Learn more about how to create an A/B test.

Baseline conversion rate

A modeled conversion rate attributed to the variant based on the percentage of sessions that resulted in a conversion (i.e. the experiment objective being met).

Combination

In a multivariate experiment, which contains multiple sections, combinations are the number of variants in each section multiplied by each other. See the example in Multivariate test. In a multivariate test with multiple sections, a combination is the experience created from each section's variants. For example, headline 1 with hero image A.

Container

A container holds all of the Optimize configuration information for a website’s experiences (tests and personalizations).

A container snippet is a small piece of JavaScript code that must be added to any web page(s) being optimized.

A container ID is a 9-digit alphanumeric string (e.g. "GTM-A1B2CD") that uniquely identifies it.

Learn more about accounts and containers.

Editor page

The web page used to create variants. Note that this may differ from your URL targeting rules.

Estimated conversion rate

(Modeled conversion rate, Projected conversion rate)

Based on the data to date, this is the conversion rate you can expect to see in the future. Because Optimize takes into account the day of week and other sections into its statistical simulations, this projected rate may differ from the historical rate measured so far.

Experiment

An experiment is used to test variants of web pages to determine which is most effective at achieving an objective. Examples include A/B, redirect, and multivariate experiments.

Experiment sessions

Any session where the experiment executed and all future sessions for that user while the experiment is running, even if they didn’t see the experiment. Subsequent sessions are included in Experiment sessions to capture conversions that occur after a user is included in the experiment. Optimize objectives that occur during an Experiment session will be included in the experiment reporting and statistics.

Learn more about sessions in the Google Analytics help center.

Fractional factorial experiment

Only a subset of combinations is served in order to find results quickly. These experiments sacrifice the detailed interaction analysis available with full-factorial experiments for greater speed in finding results.

Full factorial experiment

Every combination of a multivariate experiment is served to users, to understand all interactions between variants. These generally take longer to reach significance but also allow for detailed measurement of interactions, as well as offering the ability to serve the winning combination.

Improvement

(credible level of improvement)

The difference in the modeled conversion rate of the variant and the baseline for a given objective. This is the likely range in which your conversion rates will fall. Optimize uses Bayesian analysis to determine how the variants will perform in the future and improvement is the Bayesian equivalent of a confidence interval.

Leader

The variant that maximizes the objective.

Multivariate test

(MVT, compound test)

An experiment that tests two or more sections to understand their effects on each other. For example, variants of a headline can be tested at the same time as variants of a hero image. Instead of showing which page variant is most effective (as in an A/B test), a multivariate test identifies the most effective combination of variants. Rather than the two or three page variants found in simple A/B tests, multivariate tests frequently test multiple variants of multiple page elements simultaneously. Learn more about how to create a multivariate test.

Objective

The website functionality you wish to optimize.

Original

Your current web page, prior to any modification.

Probability to beat baseline

The probability that a given variant will result in a conversion rate better than the original's conversion rate. Note that with an original and one variant, the variant's Probability to beat baseline starts at 50 percent (which is just chance).

Probability to be best

The probability that a given variant performs better than all other variants.

Redirect destination

A separate and unique web page used as a variant in a redirect experiment.

Redirect test

(Split URL test)

A redirect test is a type of A/B test that allows you to test separate web pages against each other. In redirect tests variants are identified by URL or path instead of an element(s) on the page. Redirect tests are useful when you want to test two very different landing pages, or a complete redesign of a page. Learn more about how to create a redirect test.

Section

(Factor, Element)

A single element of a web page (e.g. a headline, image, or button) that is modified to create variants. An A/B test only contains one section (with one or more variants); in a multivariate experiment, multiple sections are tested at the same time.

Variant

(Variation, Level)

A variant can be anything from a change to a single element, or changes to multiple elements, or a totally different page in an experiment. In an A/B experiment the unit of variant can be a web page or an element of a web page. In a multivariate experiment you have multiple variant of each section.

Was this article helpful?
How can we improve it?