Ad selection process white paper

The ad selection process is the methodology that the DoubleClick for Publishers ad server uses to choose the best ad to serve when an ad is requested.

This white paper describes the tools and mechanics of the ad selection process in DoubleClick for Publishers (DFP), as well as how a creative is chosen from within a selected line item. It is intended for network administrators, traffickers, and salespeople, and assumes that you have a general understanding of the DFP system and concepts of ad serving.

 

Overview of the ad selection process

DFP's ad selection process is designed to deliver the right ad to the right customer at the right time. Below is the process that DFP follows to select an ad to serve:

FIGURE 1: Overview of the ad selection process
  1. A user's browser reads a web page that contains DFP ad tags and sends an ad request to a DFP ad server.

  2. The DFP ad server receives the ad request and gathers information to help select the best ad.

  3. The ad server creates a list of all line items matching the targeting criteria.

  4. The ad server filters out line items not eligible to serve.

  5. The ad server scans the list of eligible matching line items, starting with highest priority and working down until a line item is chosen to serve.

  6. The ad server selects the best creative from within the selected line item.

  7. The creative from the selected line item is served to the user.

The sections that follow provide detailed information about each stage in this process.

 

1. An ad tag submits a request

Tag syntax is covered separately. Learn more about generating ad tags.

 

2. Gathering information

When a user visits a web page that has DFP ad tags, the user's browser requests an ad from the DFP ad server. There are four crucial pieces of data transmitted in the ad request:

  • the HTTP header
  • the IP address
  • the DoubleClick cookie
  • the variables set by the publisher in the DFP ad tags

The following table shows how this data is used in the ad selection process. The DFP ad server only checks DoubleClick cookies if they are allowed by the individual user; that is, if the user hasn't opted out or blocked them using their browser settings.

Data Provides information on:
HTTP header Browser type
Operating system
Date and time
IP address Geographic location
Internet-related targeting information like user domain
DoubleClick cookie Frequency
Creative rotation
DFP ad tags Ad unit and size
Which creative types the ad unit can receive
Custom targeting parameters (key-values)
 

3. Creating a list of matching line items

Once the ad server gathers the relevant information for the ad request, it generates a list of all line items that match a subset of the targeting criteria of the request.

For example, if the request comes from a man in California using Linux:

  • A line item targeting Men in California will be on the list.
  • A line item targeting Men in California on Windows will not be on the list.
  • A line item targeting Men in Vermont will not be on the list.
About inventory matching: The ad server will consider all line items targeting a piece of inventory anywhere between the ad unit and the network level. This enables easy creation of run-of-site and run-of-network ads without having to remember to target every ad unit in the tree. For example, if you have a tag with a first-level ad unit called Sports and a second-level unit called Baseball, line items that are run-of-network, targeted to Sports, or targeted to Baseball could serve to that tag.
 

4. Filtering out line items that aren't eligible to serve

The ad server then removes line items from the list that aren't eligible to serve for various reasons, such as:

  • Frequency capping
  • Day parting
  • Exclusions (competitive restrictions and similar line items)

The ad server also considers factors based on previous requests on the same page, such as:

  • Roadblocking (showing two or more creatives together).
  • Creative rotation/preventing jackpotting (when the same creative serves to multiple ad units on the same page at the same time): If a creative from a line item has shown, and no other compatibly sized creatives exist in the line item, we remove the line item from the list.
 

5. Choosing the best line item to serve

Once the list of eligible line items is prepared, DFP goes through the following process to select the best one:

  1. DFP iterates through each priority level, from 1 (highest priority) to 16 (lowest priority) looking for eligible line items.
  2. Within each priority level, line items are grouped based on their goal type, and they will be served in the following order of preference until the line item's goal or cap is met:
    1. Line items with percentage goals (a specific portion of traffic, or an exclusive sale)
    2. Line items with absolute goals (impressions or clicks)
    3. Line items that are unlimited (non-guaranteed, with a click or impression cap)

To simplify the process of picking priorities and goals, DFP has eight predefined line item types from which you can chose:

Line item type Priority Goal
Sponsorship 4 percentage
Standard - high 6 absolute
Standard - medium (DEFAULT) 8 absolute
Standard - low 10 absolute
Network 12 percentage
Bulk 12 absolute
Price priority 12 unlimited
House 16 percentage

The three line item types at priority 12 illustrate the precedence of different goals. If no sponsorship or standard line items are chosen, the ad server will first look at network line items, then bulk, then price priority, even though each of these line item types is priority 12. For example, if you book a network line item at 100%, no bulk or price priority line items will serve (because the ad server serves percentage goals first, then absolute, then unlimited).

If your network has enabled AdSense or Ad Exchange to compete with your indirectly-sold line items, the DoubleClick ad server will also look for eligible AdSense or Ad Exchange line items whenever it looks at network, bulk, or price priority line items. If AdSense or Ad Exchange has a line item with a higher CPM than the one selected by DFP, regardless of the type, DFP will serve the AdSense or Ad Exchange line item. So even though a network line item may be set to 50% of impressions, for example, it could wind up not delivering if it is always beaten by AdSense or Ad Exchange.

 

For upgraded users: Unlike the DART platform, the only line item type that competes based on CPM against other line items in the DFP ad selection process is price priority. However, all line item types that compete with AdSense and AdExchange will do so based on the exact CPM entered. If no CPM is entered, a value of $0 is assumed.

 

The next section describes the ad selection process for each goal.

 

5.1. Percentage line items

Percentage (a value between 1% and 100%) is defined as the percent of targeted traffic not taken by a line item at a higher priority level. Percentage line items can be of sponsorship, network, or house type. For example:

Case one: equal priority, same percentages

  • Line item A: Targets 50% of Men in California -- Booked at priority 4
  • Line item B: Targets 50% of Men in US -- Booked at priority 4
  • The ad server adjusts so that each line item will meet its goal.

Case two: equal priority, different percentages

  • Line item A: Targets 50% of Women in New York -- Booked at priority 4
  • Line item B: Targets 25% of Women in New York -- Booked at priority 4
  • Line item A will get 50% of traffic, Line item B will get 25%; other line items at a lower priority will take the remaining 25%.

Percentages can add up to more than 100%; this is called overweighting. When line items at the same priority total more than 100%, the ad server will serve the line items in the correct porportion to each other, but you can expect each line item to underdeliver. In general, overweighting is not advised as it can lead to this type of unexpected delivery. When choosing between percentage line items at the same priority, the following logic is used:

Case one: Total percentage of line items exceeds 100% (overweighted)

  • Line item A: Percentage=50%
  • Line item B: Percentage=50%
  • Line item C: Percentage=50%
  • 1/3 serve Line item A, 1/3 serve Line item B, 1/3 serve Line item C

As you can see, since these line items are overweighted they will not be able to meet their goals of 50% each. However, they will serve evenly.

Case two: Total percentage of line items is less than 100%

  • Line item A: Percentage=25%
  • Line item B: Percentage=50%
  • 1/4 serve Line item A, 1/2 serve Line item B, 1/4 fall through to absolute or unlimited line items at this priority, or else to the next priority level.
 

5.2. Absolute goal line items

Absolute goal line items (standard high, medium, and low, along with bulk) are line items that are booked toward impression or click goals. They are chosen in a two-step process:

  1. A local decision, where each line item candidate decides whether to be considered for this request ("accept or reject").
  2. A global decision, where the ad server chooses one line item from all the candidates that accepted the local decision.

These decisions are driven by satisfaction index (SI), a measure of how far ahead or behind line item is. Line items with absolute goals can be set to three possible delivery schedules that affect SI:

  1. As fast as possible: a line item is never considered "on schedule" and tries to use every impression it can get. More about "as fast as possible" line items

    As fast as possible line items should deliver quickly if there is enough inventory, but they will never preempt "even" delivery line items that have the same priority and are further behind schedule. If an "as fast as possible line item" is the only eligible candidate, it will always serve; if there are competing line items, they will compete based on SI.

  2. Frontloaded: a line item tries to get ahead of schedule by as much as 25%.
  3. Even delivery: the ad server will deliver 5% more than the even goal. However, if your network is enabled to use expected traffic shape in pacing, the delivery indicator may be higher than 105% particularly early on in a campaign. (learn more about using expected traffic shape in pacing)

The goal SI for all three delivery schedules is the same (1,000).

SI can also be affected if your network uses expected traffic shape in pacing. If the systems knows of an upcoming traffic trend, it will give the ad server advice to adjust the delivery schedule to reflect the future trend. Hints about the shape of traffic can include trends such as low traffic on weekends or a bell curve of traffic over the course of a business day.

The local decision is made in the following way:

  1. "As fast as possible" line items will always accept.
  2. Line items that are determined to be at risk of falling behind schedule will accept.

The global decision is made in the following way:

  1. Give each line item that accepted the local decision a weight based on its SI. A line item with a lower SI will have a higher weight.
  2. Choose one line item, where each line item has the probability of being chosen in proportion with its weight.

Using weights for the global decision helps smooth out line item delivery, to avoid abrupt see-saws back and forth that would happen if behind-schedule Line item A took 100% of impressions until it was on-schedule, driving Line item B far behind schedule, causing Line item B to take 100% of impressions until it was back on schedule, inadvertently having thrown Line item A behind again, and so on.

Instead, for standard line items with a frontloaded or even delivery schedule, DFP spreads the remaining impressions over the remaining time, such that the line item will reach 100% delivery by its last day. As such, line items that are behind schedule are not treated like line items that are booked to deliver as fast as possible. Rather, they're given enough impressions to meet the delivery goal by the end of the run, while other line items with the same or lower priority are given any additional impressions. For example, if a line item that's behind schedule needs to deliver 500 impressions per day to meet its goal by the end date, but 600 impressions are available each day, other line items can be delivered for the remaining 100 impressions.

 

5.3. Unlimited line items

Unlimited line items (for example, price priority) have no goal, only a cap that limits the number of impressions or clicks the line item can receive either per day or over its lifetime. Unlimited line items are non-guaranteed because they are only considered to serve after the ad server has exhausted all percentage or absolute goal line items at the same or higher priority. If unlimited line items are being considered, the ad server chooses the line item with the highest effective CPM (eCPM), calculated in the following way:

  1. For CPM pricing, eCPM = CPM price
  2. For CPC pricing, eCPM = CPC price * historical average click-through rate * 1,000 impressions
  3. If AdSense dynamic allocation is turned on for the ad unit in the current request, AdSense's eCPM = the combined value of all the line items that AdSense would serve into this ad unit that won the AdSense auction, following all normal AdSense rules. For example, if Adsense is competing for a skyscraper and would fill the slot with five text ad units, the eCPM would be the value of the winning bid for all five line items together.

If there is more than one price priority line item at the same cost, the line items will rotate evenly.

If there are no line items available, including house line items, nothing serves (there are no system default line items). Unfilled impressions are tracked in your inventory reports (Learn more about unfilled impressions).
 

6. Choosing the best creative

Once the best line item is selected, the DFP ad server chooses the best creative in the following way:

  1. Only consider creatives that match the size of the ad request, with an allowance for slightly smaller creatives. For example, if the size in the ad request is 300x250 and the size of the creative is 300x249, the creative is still eligible to serve.

    For pages using GPT with multiple size ad slots, the slot definitions in the header should be declared according to their size, alphabetically. Learn more about slot definition and sequentiality.

  2. Only consider creatives of a valid creative type for that slot based on format (image, Flash, and so on).

  3. If there is only one matching creative in the line item, serve it.

  4. If there is more than one matching creative in the line item, follow a creative rotation rule:

    • Random: Choose a creative at random

    • Weighted: Choose a creative at random, but according to the relative weights specified for each (such as 70/30).

    • Optimized: Choose the creative with the highest historical average click-through rate.