Ad Manager Data Transfer reports

With Data Transfer reports, you can access event-level data related to your Ad Manager network. #Sample files #Data Transfer reports

Data Transfer report files provide non-aggregated, event-level data from your ad campaigns. This data is essentially raw content from the Ad Manager ad server logs, with a separate file generated for each type of event. Data Transfer files contain event data that is accurate to the second, and you can choose to include other information in the files to see device, geography, and other information related to the event. Partners need to approve ad units in Ad Manager to show in their Data Transfer files.

Only available in Google Ad Manager 360.

Get started

Data Transfer report files come at an additional cost. Contact your account manager to enable the feature. After the feature is enabled, you can set up your Data Transfer file types. 

Data Transfer requires a degree of technical expertise.
If your organization is not able to manage extract, transform and load (ETL) processing, support large files, manipulate text files, design and administer a mid-sized data store, and design and implement scripts, consider working with an approved Google Marketing Platform partner.

Available Data Transfer report files

Each Data Transfer file contains information about different events. You can add fields to each file type to see contextual information related to those events.

If there's a corresponding Backfill file, the Network file doesn't include impressions served from Ad Exchange or AdSense via dynamic allocation. Use the Backfill file for information on dynamically-allocated impressions.

About the data contained in Data Transfer files

• Late data
  Delays of a few hours are normal, but occasionally Data Transfer files take longer than usual to process. If data is late, it appears in the next hourly batched file with an accurate time stamp. This could mean, for example, that a file has mostly 8 a.m. to 9 a.m. timestamps with a scattering of earlier timestamps if processing was delayed.

• Hours with no activity
  If there is no activity during a given hour, an empty Data Transfer file that only contains header data is posted.

• Date and day boundaries
  The first hourly file for a given day typically contains events from midnight to 1 a.m. Pacific Time, but the event timestamps are in the publisher's network time zone. If, for example, the publisher is set to Eastern Time, they'd see events from 3 a.m. to 4 a.m. in the first hourly file. The three hours before that would actually be in the previous day's file. You might therefore have timestamps from a different date than is represented by the file name. Always refer to the timestamp on events in the file, not the time the file is posted or name of the file.
• Master/companion reporting in Data Transfer
  Data Transfer files show both master and companion creative impressions but the query tool only counts an impression served for the master/companion creative set (not an impression for each creative) unless information is broken down by master/companion.
  IsCompanion is “TRUE” for the Companion creative impression. The CreativeId field contains the individual creative IDs for the Master and Companion creatives and not the Creative Set ID. There isn’t an additional field in Data Transfer for Creative Set ID to associate companion impressions to master impressions.
• Discrepancies with Ad Manager reporting
  Bad traffic, or spam data, is periodically removed from Ad Manager reports or API-generated reports. Due to the publishing schedule of Data Transfer files, some of this cleanup may not be reflected. This can result in Data Transfer showing slightly more impressions, clicks, or other events. When discrepancies occur, the extent tends to be ~1%.

  Your CodeServes Data Transfer files might show a higher value than the "Code served count" metric in Ad Manager reports because Data Transfer counts a code serve for every served impression, but reports only count one code serve per request. Some creative formats, such as Google Ads text ads, can contain multiple impressions for a single request. Group your Data Transfer files by the KeyPart field values to find the unique impressions for comparison.

Download a sample file

You can download these sample files to preview the data and fields contained in each of your Data Transfer report files.

Event type	File name and type	What it shows	Sample file
Joinable Bids	NetworkBackfillBidsJoinable	Information about bids. Joinable with other Data Transfer files. 1 file per hour.	Download
CodeServe	NetworkCodeServes NetworkBackfillCodeServes	Records every response from Ad Manager, whether downloaded or not. 2 files per hour. Note: The Request and CodeServe files are included with the Impression file at no additional cost.	Download
Impression	NetworkImpressions NetworkBackfillImpressions	Information about downloaded impressions. 2 files per hour.	Download
Request	NetworkRequests NetworkBackfillRequests	Records every ad request received by Ad Manager, whether filled or unfilled. 2 files per hour. Note: The Request and CodeServe files are included with the Impression file at no additional cost.	Download
Active view	NetworkActiveViews NetworkBackfillActiveViews	Information about Active View-eligible Ad Manager-based impressions. 2 files per hour.	Download
Bids	NetworkBackfillBids (Beta)	Information about Authorized Buyers and Open Bidding auction bids. 1 file per hour.	Download
Click	NetworkClicks NetworkBackfillClicks	Information about clicks. 2 files per hour.	Download
Minimum bid to win	NetworkMinimumBidToWin (Beta)	Reports the minimum bid buyers needed to win the auction when at least one remnant line item is competing. 1 file per hour.	Download
Rich Media conversion	NetworkRichMediaConversions NetworkBackfillRich MediaConversions	Information about Studio events, including both standard and custom actions (like, play and pause), action duration, and more. 2 files per hour.	Download
Video conversion	NetworkVideoConversions NetworkBackfillVideo Conversions	Information about video-specific events, including actions (like, play and pause), content IDs, pod positioning, and more. See all video events. 2 files per hour.	Download

How files are delivered

Data Transfer files are pushed to Ad Manager cloud storage buckets on an hourly basis. We advise polling at regular intervals to check for updates. The majority of the data is delivered and available between 5 and 15 hours after the recorded hour, though some delayed events can take up to 8 days after the event occurred. Ad Manager does not deliver data transfer information to third-party servers.

Daylight saving time
File names include the start hour for events in US Pacific time zone (observing Daylight saving time), but the timestamps present in that file are always given according to the Ad Manager network time zone (which might not observe Daylight saving time). This can lead to empty/skipped files or files containing more than one hour's worth of data, depending on the interplay between these time zone settings.

 

All impression data is delivered during the Daylight saving time cutover.

File names

Data Transfer file names follow a predictable convention: [Type]_[Network ID]_[YYYYMMDD]_[HH].gz

• YYYYMMDD is the year, month, and date.
• HH is the start hour in 24-hour format.

  The hour number (01, 02, 03) specified in each file name is in Pacific timezone, but publishers get their own network timezone-specific data from the timestamps contained within the Data Transfer files. Be aware of this difference when you calculate file delivery.

In extremely rare cases, Data Transfer files can be re-published to correct erroneous data. In these cases, the re-published files have the string "_corrected" appended to the file name, but the previous versions of the files are not deleted from the bucket. 

Data Transfer files in the ad request process

Review the following graphic to understand the Data Transfer file types associated with various stages of the ad request process.

Use Data Transfer report files

Once you've set up Data Transfer, files are kept in Ad Manager cloud storage buckets. You can access them on the web, with a command line tool, or through an API. Learn more about how to access Ad Manager cloud storage buckets

Use the timestamps on each row in the files instead of the hour listed in the file name
If you limit your data ingestion and analysis to a specific set of Data Transfer files based on the start hour in the file name, you might overlook data that's provided in a subsequent file because of daylight savings time, late data collection, or other similar scenarios. A better approach is to read all the Data Transfer files into a separate system (such as a data warehouse or query engine) and restrict your analysis based on the timestamp of the events.

Data Transfer files are saved as .CSV files that you can convert using a spreadsheet editor. Here are some examples of typical ways you may choose to apply Data Transfer information:

• Dimensional user aggregation of events and activities (for example, per creative)
• Calculate unique conversions across multiple days
• Match users against a customer database
• Report on user geographic and demographic information

You can use match tables to provide a name-to-ID lookup for values contained within Data Transfer files, allowing you to match ad serving information (such as ad unit or line item) to pre-assigned values stored in the database.

Store files locally

A good rule of thumb is that each event uses between 25 and 35 bytes in a compressed file. As such, 10 million impressions would require about 300 MB of disk space in a compressed file. Keep in mind that these are estimates, and your file size could be somewhat larger. Also, because this is the size of the data in compressed form, you need additional space to decompress and use the files.

Data Transfer files older than 60 days are purged from Ad Manager. If you would like to store your files for longer than the allotted 60 days, we recommend that you either store the files locally or move to a permanent cloud storage solution, which can include an independent Google Cloud Storage account over which you have full control.

Learn about the BigQuery Data Transfer Service

You can use the BigQuery Data Transfer Service with Data Transfer reporting. For details and instructions, go to Google Ad Manager Transfers.

Make large Data Transfer files easier to process

Google Code has released CRUSH (Custom Reporting Utilities for Shell), an open-source toolkit for processing delimited-text data from the command line or in shell scripts. The CRUSH tools have been extensively developed and tested, and they work best on Linux or Unix operating systems. Support for CRUSH is available through the open-source community.

A non-open-source alternative is DMX, a data integration software developed by Syncsort.