Batch upload identifiers to audience segments

This article is specific to Audience Solutions.
This feature might not be enabled in your network.

If you need to add many identifiers (cookies, mobile advertising IDs (AdID or IDFA), or PPIDs) to a first-party audience segment, you can save time by creating a file of IDs and uploading it into Google Ad Manager via the Google cloud. Batch upload does not work with third-party segments.

Cookies, PPIDs, and device advertising IDs must have been part of an Ad Manager ad request before they can be added to a segment. You can download lists of encrypted identifiers in Data Transfer reports (Google Ad Manager 360 only), or retrieve raw identifiers from other systems (e.g., raw device advertising IDs from your mobile application's logs).

This article covers the following topics:

Prerequisites
Create the upload file
Deleting identifiers in bulk
Upload the file
Batch uploaded identifier expiration
Best practices for batch uploads

Prerequisites

Before you can upload the identifiers mentioned above, you'll need work with your account manager to:

  • Locate and submit your network code. Find your network code in Ad Manager under Admin and then Global settings and then Network code.
  • Ensure that you've created an active first-party audience segment with which you want to associate the identifiers. If you don't want this segment to collect users using your own inventory, mark the Audience pixel segment option.
  • Create a Google group that contains all Google Accounts that will have access to upload and view files.

Once your account manager has completed your setup, you'll be ready to create the file of identifiers you want to upload.

Create the upload file

About identifiers

The batch upload process supports several different identifier types, and it's important that types of identifiers are uploaded to the segment using the correct upload file format. Broadly, these identifiers fall into one of two categories: encrypted identifiers (anything obtained from Google systems), and raw identifiers (obtained from an external system or source). Any encrypted identifier is uploaded using the cookie_encrypted file format, while raw identifiers are uploaded in a type-specific upload file format.

Encrypted identifiers and PPIDs are network-specific—you cannot use batch upload to add PPIDs or encrypted identifiers from one network into another network. For instance, a user identifier obtained from network A's Data Transfer report (Google Ad Manager 360 only) cannot be batch uploaded into network B. Device identifiers are not network-specific.
The following table summarizes the available identifiers and how to upload:

 

Identifier Source Upload as...
Cookie Data transfer
UserId
cookie_encrypted
Raw (external source) N/A (cookie must be encrypted to upload)
PPID Data transfer
PublisherProvidedID
cookie_encrypted
Raw (external source) ppid
IDFA Data transfer
UserId
cookie_encrypted
Raw (external source) cookie_idfa
AdID Data transfer
UserId
cookie_encrypted
Raw (external source) cookie_adid
Roku ID Data transfer
UserId
cookie_encrypted
Raw (external source) cookie_rida
tvOS ID Data transfer
UserId
cookie_encrypted
Raw (external source) cookie_tvos

Using a text editor, create a plain text file and save it using any file name. You can save the file using any file extension or no extension at all.

Google cloud file names are publicly visible. Do not include confidential information in your file names.

File content requirements

  • Ensure that the contents of the file are comma-separated.
  • The first line must read:
    • For encrypted cookies and encrypted identifiers obtained from Data Transfer reports (Google Ad Manager 360 only): cookie_encrypted,list_id
    • For raw Apple IDFAs: cookie_idfa,list_id
    • For raw Android AdIDs: cookie_adid,list_id
    • For raw PPIDs: ppid,list_id
    • For raw Roku IDs: cookie_rida,list_id
    • For raw tvOS IDs: cookie_tvos,list_id
  • Each subsequent line should contain an identifier, a comma, and the ID of the audience segment with which the identifier should be associated
  • Each line should end with the unix line ending type linefeed (LF), NOT carriage return (CR). Depending on the method you use to produce the upload file, the line ending type is set in your text editor or in the program that produces the file.
  • The encrypted identifiers are available to publishers through a Data Transfer report (Google Ad Manager 360 only). You can get the audience segment ID by signing into your Google Ad Manager account and selecting Inventory and then Audience segments.
  • Batch upload files cannot be larger than 1GB in size. Files larger than this might fail to process correctly.

Example file content for encrypted cookie IDs:

cookie_encrypted,list_id
ScpJKu-yV8je93qkd32MOA,3153490
w2gsrUcwxF-OiJTRmQswQA,3153490

 

Example file content for raw AdIDs:

cookie_adid,list_id
38400000-8cf0-11bd-b23e-10b96e40000d,3153490
38400001-8cf1-11be-b23f-10b96e40000e,3153490

Deleting identifiers in bulk

You can use the same upload file to delete identifiers in bulk by adding a column called delete. For each identifier you want to delete from the segment, enter a 1 in the delete column and for each ID you want to keep in the segment, enter a 0 in the delete column.

You must ensure that every line in the file contains either a 1 or a 0 in the delete column. Otherwise, the upload will fail.

In the example below, the first cookie is kept in the segment, but the second cookie is disassociated from the segment with id=3153490.

cookie_encrypted,list_id,delete
ScpJKu-yV8je93qkd32MOA,3153490,0
w2gsrUcwxF-OiJTRmQswQA,3153490,1

In the following example, the first IDFA is disassociated from the segment, but the second IDFA is kept in the segment.

cookie_idfa,list_id,delete
AEBE52E8-03EF-455B-B3C5-E57283966240,3153490,1
AEBE52E9-03F0-455C-B3C6-E57283966241,3153490,0

Each line must contain as many comma-separated fields as a present in the header. Note that the identifiers never contain commas or other characters inconsistent with CSV format.

Keep in mind that it can take several hours for Ad Manager to process identifier uploads, whether you're adding or deleting them. When you add an identifier, you need to wait for it to be processed before you can delete it with a subsequent upload.

Upload the file

Once you've created the file, you'll need to upload it to a Google Storage bucket (subfolders are allowed). There are three ways you can access Ad Manager cloud storage buckets. In order of complexity:

  • On the web: Visit https://console.developers.google.com/storage/gdfp_cookieupload_[Ad Manager network code]/.
  • gsutil is a Python-based command-line tool that provides Unix-like commands for interacting with the storage bucket. Bucket authentication is abstracted and handled automatically.
  • The Google Cloud Storage API is a full-featured API for manipulating the storage bucket, available through JSON or XML RESTful web interfaces. API client libraries are available for many popular programming environments, including Java, JavaScript, Python, and Objective-C. This approach is most useful if you need to manipulate the storage buckets programmatically to integrate with a Google App Engine app or a Java web app.

Within 24 hours, a log file will be generated in your Google Storage account with the same name as your file and a suffix indicating whether all of the identifiers in the file were processed successfully. For example, if your file is named 1234_20130115_1, then the generated file will be named 1234_20130115_1.success or 1234_20130115_1.fail. It lists the number of identifiers processed successfully, with detailed error messages for identifiers that could not be processed. Download this log file for more detailed status information.

The results filename will appear in the following folder of your Google Storage account:

https://console.developers.google.com/storage/gdfp_cookieupload_result_[Ad Manager network code]/

After upload, the successfully processed identifiers in your file are treated as members of their associated segments when targeting inventory.

The .fail or .success file only indicates whether the .dat file was parsed successfully; it doesn't provide meaningful information about the net increase/decrease in list size. The file is only named .success if every single identifier was uploaded successfully. However, in almost all cases, there's at least one failure, so the output file has the .fail suffix.

The upload and log files will be deleted after 60 days.

Batch uploaded identifier expiration

Identifiers uploaded as part of a Batch Upload operation obey the expiration settings that have been configured on the audience segment through the Ad Manager UI. If the audience segment is configured with an expiration value of X days and the identifier does not satisfy the segment membership criteria within X days after upload, that identifier will expire from the audience segment after X days. Each upload operation resets the segment membership and expiration countdown for the identifiers that are uploaded.

Best practices for batch uploads

File structure

Minimize the number of files used for uploads. Note that a single file can be used to upload identifiers into multiple lists, since the list id is specified for each line in the file.

Upload throughput

We request you adhere to the following limits when uploading files:

  • Upload no more than one batch upload file per hour
  • Up to 1 GB per batch upload file
  • Up to 5 GB in total batch upload files per 24 hour window

Incremental/complete uploads

  • To avoid unnecessary processing, we recommend using an incremental (delta) upload of identifiers as a routine practice. You should add or delete only the identifiers whose membership in the segment has changed since the last upload, rather than uploading the full segment identifier list.
  • On a less frequent basis (no more than once per week, and still subject to the upload throughput limits above), you may run a full upload of all segment identifier data. This full upload will refresh the segment membership of any members who had previously aged out of the audience segment.
    • You should be tracking the "freshness" of identifiers through some other system, and you should have some way to exclude identifiers from the refresh upload if these identifiers have been abandoned (are not expected to be seen again in serving).

Processing time

Expect to wait up to 24 hours from the time of batch upload to the time when all batch changes are reflected in serving. In other words, upload your files at least 24 hours ahead of when you need the segments in serving. 

Was this helpful?
How can we improve it?