Search
Clear search
Close search
Google apps
Main menu

Export billing data to BigQuery

Tools for monitoring, analyzing and optimizing cost have become an important part of managing development. Billing export to Google BigQuery enables you to export your daily usage and cost estimates automatically throughout the day to a BigQuery dataset you specify. You can then access your billing data from BigQuery. You can also use this export method to export data to a JSON file.

Note: Regular file export to CSV and JSON is also available. However, if you use regular file export, you should be aware that regular file export captures a smaller dataset than export to BigQuery. For more information about regular file export and the data it captures, see Contents of the exported billing file.

How to enable billing export to BigQuery

To enable billing export to BigQuery:

  1. Go to the Cloud Platform Console.
  2. Open the left side menu and select Billing  Billing
  3. If you have more than one billing account, select the name of the account you're interested in.
  4. Click Billing export.
  5. Select BigQuery export.
  6. From the Project list, select the project where your BigQuery dataset is stored. If you don’t have a BigQuery dataset created yet, you’ll be prompted to create one.

    If necessary, to create a new dataset:

    1. In the BigQuery web UI, click the down arrow icon next to your project name, then click Create new dataset.
    2. Specify the Dataset ID, Data location, and Data expiration for your dataset, then click OK.
  7. In the console, from the Billing export dataset list, specify a dataset to export data to. If you just created a dataset, choose the name of your new dataset.
  8. Click Enable BigQuery export.

After you enable BigQuery export, it might take a few hours to start seeing your data. Billing data automatically exports your data to BigQuery in regular intervals, but the frequency of updates in BigQuery varies depending on the services you're using.

You should be aware that in the unlikely event that your billing data contains an error or requires an adjustment, the data might be overwritten. If you need to check the date that the data was updated, you can do so by using a query like the following example:
 
SELECT partition_id, MSEC_TO_TIMESTAMP(last_modified_time)
FROM [dataset_name.table_name$__PARTITIONS_SUMMARY__]

 
You must use legacy SQL because standard SQL does not support the partition decorator separator ($).

 

Important notes: The billing export table is partitioned by date, but note that this is an internal processing date and not equal to the usage date.

For descriptions of the contents of the exported billing data, see the following section, Billing data in BigQuery.

Billing data in BigQuery

The following table describes the contents of the billing data that is exported to Google BigQuery.

For information about how to enable automatically exporting billing data to BigQuery, see the instructions above.

Note: Exported billing data does not include any tax accrued or credits issued to a billing account.

Field Type Description
billing_account_id String The billing account ID that the usage is associated with.
product String The Google Cloud Platform service that reported the billing data.
resource_type String A description of the resource used by the service. For example, a resource type for Cloud Storage is Standard Storage US.
start_time Timestamp

A UNIX timestamp of when the service usage began. For more information, see the BigQuery documentation on timestamp data types.

Note: This date will not necessarily match the partition date, which is an internal processing date.

end_time Timestamp

A UNIX timestamp of when the service usage ended. For more information, see the BigQuery documentation on timestamp data types.

Note: This date will not necessarily match the partition date, which is an internal processing date.

project.id String The ID of the project that generated the billing data.
project.name String The name of the project that generated the billing data.
project.labels.key String If labels are present, the key portion of the key:value pair that comprises the label on the project where the usage occurred. For more information about using labels, see Using Labels.
project.labels.value String If labels are present, the value portion of the key:value pair that comprises the label on the project where the usage occurred. For more information about using labels, see Using Labels.
labels.key String If labels are present, the key portion of the key:value pair that comprises the label on the resource where the usage occurred. For more information about using labels, see Using Labels.
labels.value String If labels are present, the value portion of the key:value pair that comprises the label on the resource where the usage occurred. For more information about using labels, see Using Labels.
cost Float The amount charged to your billing account.
currency String The currency that the cost is billed in. For more information, see Local currency billing and payments.
currency_conversion_rate Float The exchange rate from US dollars to the local currency. That is, cost/currency_conversion_rate is the cost in US dollars.

Note: When Google charges in local currency, we convert prices into applicable local currency pursuant to the conversion rates published by leading financial institutions.

usage.amount Float The quantity of usage.units used.
usage.unit String The measurement of resource used. For example, a usage unit for standard storage is byte-seconds.
credits.name String A description of the credit applied to the billing account.
credits.amount Float The amount of the credit applied to the usage. Credits are always negative amounts and the total charge is calculated as cost + credits.amount.

 

About labels: You'll see columns for labels in your BigQuery dataset, but for the current release some label values will be empty. Label export data will be populated at different times for different services, depending on when each service provides it.

Using Google BigQuery to store usage data will incur minimal fees. For more information, see Google BigQuery Pricing.

Note: This feature is in preview. The formatting and content of the dataset might change.

Was this article helpful?
How can we improve it?