Export billing data to BigQuery
Tools for monitoring, analyzing and optimizing cost have become an important part of managing development. Billing export to Google BigQuery enables you to export your daily usage and cost estimates automatically throughout the day to a BigQuery dataset you specify. You can then access your billing data from BigQuery. You can also use this export method to export data to a JSON file.
Note: Regular file export to CSV and JSON is also available. However, if you use regular file export, you should be aware that regular file export captures a smaller dataset than export to BigQuery. For more information about regular file export and the data it captures, see Contents of the exported billing file.How to enable billing export to BigQuery
To enable billing export to BigQuery:
- Go to the Cloud Platform Console.
- Open the left side menu and select Billing
- If you have more than one billing account, select the name of the account you're interested in.
- Click Billing export.
- Select BigQuery export.
From the Project list, select the project where your BigQuery dataset is stored. If you don’t have a BigQuery dataset created yet, you’ll be prompted to create one.
If necessary, to create a new dataset:
- In the BigQuery web UI, click the down arrow icon next to your project name, then click Create new dataset.
- Specify the Dataset ID, Data location, and Data expiration for your dataset, then click OK.
- In the console, from the Billing export dataset list, specify a dataset to export data to. If you just created a dataset, choose the name of your new dataset.
- Click Enable BigQuery export.
After you enable BigQuery export, it might take a few hours to start seeing your data. Billing data automatically exports your data to BigQuery in regular intervals, but the frequency of updates in BigQuery varies depending on the services you're using.
You should be aware that in the unlikely event that your billing data contains an error or requires an adjustment, the data might be overwritten. If you need to check the date that the data was updated, you can do so by using a query like the following example:
SELECT partition_id, MSEC_TO_TIMESTAMP(last_modified_time)
You must use legacy SQL because standard SQL does not support the partition decorator separator
For descriptions of the contents of the exported billing data, see the following section, Billing data in BigQuery.
The following table describes the contents of the billing data that is exported to Google BigQuery.
For information about how to enable automatically exporting billing data to BigQuery, see the instructions above.
||String||The billing account ID that the usage is associated with.|
||String||The Google Cloud Platform service that reported the billing data.|
||String||A description of the resource used by the service. For example, a resource type for Cloud Storage is Standard Storage US.|
||Timestamp||A UNIX timestamp of when the service usage began. For more information, see the BigQuery documentation on timestamp data types.|
||Timestamp||A UNIX timestamp of when the service usage ended. For more information, see the BigQuery documentation on timestamp data types.|
||String||The ID of the project that generated the billing data.|
||String||The name of the project that generated the billing data.|
||String||If labels are present, the key portion of the key:value pair that comprises the label on the project where the usage occurred. For more information about using labels, see Using Labels.|
||String||If labels are present, the value portion of the key:value pair that comprises the label on the project where the usage occurred. For more information about using labels, see Using Labels.|
||String||If labels are present, the key portion of the key:value pair that comprises the label on the resource where the usage occurred. For more information about using labels, see Using Labels.|
||String||If labels are present, the value portion of the key:value pair that comprises the label on the resource where the usage occurred. For more information about using labels, see Using Labels.|
||Float||The amount charged to your billing account.|
||String||The currency that the cost is billed in. For more information, see Local currency billing and payments.|
||Float||The exchange rate from US dollars to the local currency. That is,
Note: When Google charges in local currency, we convert prices into applicable local currency pursuant to the conversion rates published by leading financial institutions.
||Float||The quantity of
||String||The measurement of resource used. For example, a usage unit for standard storage is byte-seconds.|
||String||A description of the credit applied to the billing account.|
||Float||The amount of the credit applied to the usage. Credits are always negative amounts and the total charge is calculated as
About labels: You'll see columns for labels in your BigQuery dataset, but for the current release some label values will be empty. Label export data will be populated at different times for different services, depending on when each service provides it.
Using Google BigQuery to store usage data will incur minimal fees. For more information, see Google BigQuery Pricing.
Note: This feature is in preview. The formatting and content of the dataset might change.