Ad selection process white paper
The ad selection process is the methodology that the DoubleClick for Publishers ad server uses to choose the best ad to serve when an ad is requested.
This white paper describes the tools and mechanics of the ad selection process in DoubleClick for Publishers (DFP), as well as how a creative is chosen from within a selected line item. It is intended for network administrators, traffickers, and salespeople, and assumes that you have a general understanding of the DFP system and concepts of ad serving.
Overview of the ad selection process
DFP's ad selection process is designed to deliver the right ad to the right customer at the right time. Below is the process that DFP follows to select an ad to serve:FIGURE 1: Overview of the ad selection process
- A user's browser reads a web page that contains DFP ad tags and sends an ad request to a DFP ad server.
- The DFP ad server receives the ad request and gathers information to help select the best ad.
- The ad server creates a list of all line items matching the targeting criteria.
- The ad server filters out line items not eligible to serve.
- The ad server scans the list of eligible matching line items, starting with highest priority and working down until a line item is chosen to serve.
- The ad server selects the best creative from within the selected line item.
- The creative from the selected line item is served to the user.
The sections that follow provide detailed information about each stage in this process.
Tag syntax is covered separately. Learn more about generating ad tags.
When a user visits a web page that has DFP ad tags, the user's browser requests an ad from the DFP ad server. There are four crucial pieces of data transmitted in the ad request:
- the HTTP header
- the IP address
- the DoubleClick cookie
- the variables set by the publisher in the DFP ad tags
The following table shows how this data is used in the ad selection process. The DFP ad server only checks DoubleClick cookies if they are allowed by the individual user; that is, if the user hasn't opted out or blocked them using their browser settings.
|Data||Provides information on:|
|HTTP header||Browser type|
|Date and time|
|IP address||Geographic location|
|Internet-related targeting information like user domain|
|DFP ad tags||Ad unit and size|
|Which creative types the ad unit can receive|
|Custom targeting parameters (key-values)|
Once the ad server gathers the relevant information for the ad request, it generates a list of all line items that match a subset of the targeting criteria of the request.
For example, if the request comes from a man in California using Linux:
- A line item targeting Men in California will be on the list.
- A line item targeting Men in California on Windows will not be on the list.
- A line item targeting Men in Vermont will not be on the list.
The ad server then removes line items from the list that aren't eligible to serve for various reasons, such as:
- Frequency capping
- Day parting
- Exclusions (competitive restrictions and similar line items)
The ad server also considers factors based on previous requests on the same page, such as:
- Roadblocking (showing two or more creatives together).
- Creative rotation/preventing jackpotting (when the same creative serves to multiple ad units on the same page at the same time): If a creative from a line item has shown, and no other compatibly sized creatives exist in the line item, we remove the line item from the list.
Once the list of eligible line items is prepared, DFP goes through the following process to select the best one:
- DFP iterates through each priority level, from 1 (highest priority) to 16 (lowest priority) looking for eligible line items.
On rare occasions, other factors such as pCTR, Ad Exchange revenue, and dynamic allocation can outweigh the value of priority when determining which line items DFP selects.
- Within each priority level, line items are grouped based on their goal type, and they will be served in the following order of preference until the line item's goal or cap is met:
To simplify the process of picking priorities and goals, DFP has eight predefined line item types from which you can chose:
|Line item type||Priority||Goal|
|Standard - high||6||absolute|
|Standard - normal (DEFAULT)||8||absolute|
|Standard - low||10||absolute|
The three line item types at priority 12 illustrate the precedence of different goals. If no sponsorship or standard line items are chosen, the ad server will first look at network line items, then bulk, then price priority, even though each of these line item types is priority 12. For example, if you book a network line item at 100%, no bulk or price priority line items will serve (because the ad server serves percentage goals first, then absolute, then unlimited).
If your network is enabled for enhanced dynamic allocation, DFP optimizes the distribution of Ad Exchange and AdSense impressions throughout goal-based line item delivery, without compromising reservation goals. Specifically, with enhanced dynamic allocation, DFP considers standard line items and remnant line items that can serve to an eligible impression. It then makes its selection based on the higher of the CPM of eligible remnant line items or a calculated reserve price of line items with delivery goals. Find out more about the dynamic allocation ad selection process.
The next section describes the ad selection process for each goal.
Percentage (a value between 1% and 100%) is defined as the percent of targeted traffic not taken by a line item at a higher priority level. Percentage line items can be of sponsorship, network, or house type. For example:
Case one: equal priority, same percentages
- Line item A: Targets 50% of Men in California -- Booked at priority 4
- Line item B: Targets 50% of Men in US -- Booked at priority 4
- The ad server adjusts so that each line item will meet its goal.
Case two: equal priority, different percentages
- Line item A: Targets 50% of Women in New York -- Booked at priority 4
- Line item B: Targets 25% of Women in New York -- Booked at priority 4
- Line item A will get 50% of traffic, Line item B will get 25%; other line items at a lower priority will take the remaining 25%.
Percentages can add up to more than 100%; this is called overweighting. When line items at the same priority total more than 100%, the ad server will serve the line items in the correct proportion to each other, but you can expect each line item to underdeliver. In general, overweighting is not advised as it can lead to this type of unexpected delivery. When choosing between percentage line items at the same priority, the following logic is used:
Case one: Total percentage of line items exceeds 100% (overweighted)
- Line item A: Percentage=50%
- Line item B: Percentage=50%
- Line item C: Percentage=50%
- 1/3 serve Line item A, 1/3 serve Line item B, 1/3 serve Line item C
As you can see, since these line items are overweighted they will not be able to meet their goals of 50% each. However, they will serve evenly.
Case two: Total percentage of line items is less than 100%
- Line item A: Percentage=25%
- Line item B: Percentage=50%
- 1/4 serve Line item A, 1/2 serve Line item B, 1/4 fall through to absolute or unlimited line items at this priority, or else to the next priority level.
Absolute goal line items (standard high, medium, and low, along with bulk) are line items that are booked toward impression or click goals. They are chosen in a two-step process:
- A local decision, where each line item candidate decides whether to be considered for this request ("accept or reject").
- A global decision, where the ad server chooses one line item from all the candidates that accepted the local decision.
These decisions are driven by satisfaction index (SI), a measure of how far ahead or behind line item is. Line items with absolute goals can be set to three possible delivery schedules that affect SI:
- As fast as possible: a line item is never considered "on schedule" and tries to use every impression it can get. More about "as fast as possible" line items
As fast as possible line items should deliver quickly if there is enough inventory, but they will never preempt "even" delivery line items that have the same priority and are further behind schedule. If an "as fast as possible line item" is the only eligible candidate, it will always serve; if there are competing line items, they will compete based on SI.
- Frontloaded: a line item tries to get ahead of schedule by as much as 25%.
- Even delivery: the ad server will deliver 5% more than the even goal. However, if your network is enabled to use expected traffic shape in pacing, the delivery indicator may be higher than 105% particularly early on in a campaign. (learn more about using expected traffic shape in pacing)
The goal SI for all three delivery schedules is the same (1,000).
The local decision is made in the following way:
- "As fast as possible" line items will always accept.
- Line items that are determined to be at risk of falling behind schedule will accept.
The global decision is made in the following way:
- Give each line item that accepted the local decision a weight based on its SI. A line item with a lower SI will have a higher weight.
- Choose one line item, where each line item has the probability of being chosen in proportion with its weight.
Using weights for the global decision helps smooth out line item delivery, to avoid abrupt see-saws back and forth that would happen if behind-schedule Line item A took 100% of impressions until it was on-schedule, driving Line item B far behind schedule, causing Line item B to take 100% of impressions until it was back on schedule, inadvertently having thrown Line item A behind again, and so on.
Instead, for standard line items with a frontloaded or even delivery schedule, DFP spreads the remaining impressions over the remaining time, such that the line item will reach 100% delivery by its last day. As such, line items that are behind schedule are not treated like line items that are booked to deliver as fast as possible. Rather, they're given enough impressions to meet the delivery goal by the end of the run, while other line items with the same or lower priority are given any additional impressions. For example, if a line item that's behind schedule needs to deliver 500 impressions per day to meet its goal by the end date, but 600 impressions are available each day, other line items can be delivered for the remaining 100 impressions.
Unlimited line items (for example, price priority) have no goal, only a cap that limits the number of impressions or clicks the line item can receive either per day or over its lifetime. Unlimited line items are non-guaranteed because they are only considered to serve after the ad server has exhausted all percentage or absolute goal line items at the same or higher priority. If unlimited line items are being considered, the ad server chooses the line item with the highest effective CPM (eCPM), calculated in the following way:
- For CPM pricing, eCPM = CPM price
- For CPC pricing, eCPM = CPC price * historical average click-through rate * 1,000 impressions
- If AdSense dynamic allocation is turned on for the ad unit in the current request, AdSense's eCPM = the combined value of all the line items that AdSense would serve into this ad unit that won the AdSense auction, following all normal AdSense rules. For example, if Adsense is competing for a skyscraper and would fill the slot with five text ad units, the eCPM would be the value of the winning bid for all five line items together.
If there is more than one price priority line item at the same cost, the line items will rotate evenly.
Once the best line item is selected, the DFP ad server chooses the best creative in the following way:
- Only consider creatives that match the size of the ad request, with an allowance for slightly smaller creatives. For example, if the size in the ad request is 300x250 and the size of the creative is 300x249, the creative is still eligible to serve.
For pages using GPT with multiple size ad slots, the slot definitions in the header should be declared according to their size, alphabetically. Learn more about slot definition and sequentiality.
- Only consider creatives of a valid creative type for that slot based on format (image, Flash, and so on).
- If there is only one matching creative in the line item, serve it.
- If there is more than one matching creative in the line item, follow a creative rotation rule:
- Random: Choose a creative at random
- Weighted: Choose a creative at random, but according to the relative weights specified for each (such as 70/30).
- Optimized: Choose the creative with the highest historical average click-through rate.