Glossary of terms

The language, lexicon and lingo of testing.

In order to get the most out of Optimize, it’s important to understand a few concepts and terms.

A/B test

(A/B/n test, experiment)

A randomized experiment using two or more variants of the same web page (A and B). Variant A is the original and variant B through n each contain at least one element that is modified from the original.

Learn more about how to create an A/B test.

Combination

In a multivariate test (MVT) with multiple sections, a combination is a set of changes created by selecting a single variant from each section. The total number of combinations is the product of the number of variants in each section.

See also: Multivariate test.

Container

(Container snippet, container ID, measurement ID)

A container holds all of the Optimize configuration information for a website’s experiences (tests and personalizations). A "container snippet" is a small piece of JavaScript code that's added to a web page where you want to use Optimize to present users with a new experience, while a "container ID" is an alphanumeric string (e.g. "GTM-A1B2CD") that uniquely identifies it. You will also see container ID referred to as "OPT_MEASUREMENT_ID" in code samples.

Learn more about accounts and containers.

Conversion

(from GA Conversion)

A completed activity that is important to the success of your business. Examples include a completed sign-up for your email newsletter (a Goal conversion) and a purchase (a transaction, sometimes called an Ecommerce conversion).

See also: Conversion rate, Experiment conversions, Calculated conversion rate.

Conversion rate

(from GA Conversion rate)

The conversion rate is the number of objective conversions divided by the total number of sessions.

Calculated conversion rate

(CCR, observed conversion rate)

Experiment Conversions divided by Experiment Sessions expressed as a percentage.

Editor page

The web page used as a template when defining the changes for a variant. Note: this may differ from your URL targeting rules.

Experiment

(Test, experience)

An experiment is a method of testing different variants of web pages to determine which is most effective at achieving an objective. Examples include A/B, redirect, and multivariate tests.

Experiment sessions

All sessions attributed to the experiment. User attribution starts when the experiment is shown to the user and ends when the experiment ends. All sessions during this period, whether the experiment was applied or not, will be counted.

Experiment conversions

Conversions in any experiment session excluding conversions in the first session per user that occur prior to the experiment being shown.

Fractional-factorial experiment

When only a subset of all possible combinations are served in order to find results quickly. These experiments sacrifice the detailed interaction analysis available with full-factorial experiments for greater speed in finding results.

Full-factorial experiment

Every combination of a multivariate experiment is served to users, to understand all interactions between variants. These generally take longer to reach significance but also allow for detailed measurement of interactions.

Goals

(Google Analytics goals)

See Objectives.

Leader

The variant that performs best against the objective.

Modeled conversion rate

(MCR, estimate conversion rate, projected conversion rate)

The conversion rate you can expect to see over the long run based on data observed to date. Optimize considers day of week (and other signals) in its statistical simulations so the modeled conversion rate may differ from the historical conversion rate measured to date.

Modeled improvement

(MI, credible level of improvement)

The relative difference between the conversion rate of the variant and the original for a given objective. The credible interval is the likely range the ratio between the conversion rate of the variant and the original will fall. Optimize uses Bayesian analysis to determine how the variants will perform over the long run and the credible interval is the Bayesian analogue of a confidence interval.

Multivariate test

(MVT, compound test)

An experiment that tests two or more sections to understand their effects on each other. For example, variants of a headline can be tested at the same time as variants of a hero image. Instead of showing which page variant is most effective (as in an A/B test), a multivariate test identifies the most effective combination of variants. Rather than the two or three page variants found in simple A/B tests, multivariate tests frequently test multiple variants of multiple page elements simultaneously. Learn more about how to create a multivariate test.

Objective

(System objectives, Analytics goals, and custom objectives)

An objective is the website metric you wish to optimize. Think of objectives as the metrics or activities that you'll measure your variants against, e.g. pageviews, bounce rate, time on site, etc. There are three types of objectives in Optimize: system objectives, Analytics goals, and custom objectives. Learn more about objectives.

Observed data

Observed data is sourced from the linked Google Analytics property, as of the date in the timestamp in the report.

Seen in: Optimize reports.

Optimize analysis

Optimize analysis includes Probability to be best (PBB), Probability to beat original (PBO), Modeled conversion rate (MCR), and Modeled improvement (MI).

Seen in: Optimize reports.

Original

(Baseline)

Your current web page, prior to any modification.

Personalization

A personalization is a set of changes made to your website for a specific group of visitors. Unlike experiments, personalizations can run forever and don't have variants. A personalization is a single set of website changes that are shown to all visitors that meet the targeting conditions.

Probability to beat original

(PBO, probability to beat baseline)

The probability that a given variant will result in a better conversion rate than the original.

Probability to be best

(PBB)

The probability that a given variant performs better than all other variants.

Redirect destination

A separate and unique web page used as a variant in a redirect experiment.

Redirect test

(Split URL test)

A redirect test is a type of A/B test that allows you to test separate web pages against each other. In redirect tests variants are identified by URL or path instead of an element(s) on the page. Redirect tests are useful when you want to test two very different landing pages, or a complete redesign of a page. Learn more about how to create a redirect test.

Section

(Factor, element)

A section is a group of variants, each of which can modify multiple page elements (e.g. a headline, image, or button). An A/B test only contains one section (with one or more variants), while a multivariate test (MVT) tests multiple sections simultaneously.

Session

(from: GA: Session)

The period of time a user is active on your site or app. By default, if a user is inactive for 30 minutes or more, any future activity is attributed to a new session. Users that leave your site and return within 30 minutes are counted as part of the original session.

Targeting rules

(URL targeting rules, rules)

Rules are the building blocks of Optimize targeting. An experiment will only execute when its targeting rules are met. Rules can be used to target everything from geographic regions to specific user behavior.

Variant

(Variation, level)

A variant can be anything from a change to a single element, or changes to multiple elements, or a totally different page in an experiment. In an A/B experiment the unit of variant can be a web page or an element of a web page. In a multivariate experiment you have multiple variant of each section.

Was this helpful?
How can we improve it?