Introduction
This document describes how to configure Google Global Cache (GGC) nodes to operate with your Carrier Grade NAT (CGN) systems. Once configured, this allows traffic between the GGC node and private user IPs to bypass the CGN.
GGC has no requirement that "private IP space" be in the RFC1918 allocation. Any non-internet-routed space in use in your network can bypass a CGN with this configuration. If appropriate for your usage, we recommend using the dedicated '100.64.0.0/10' CGN space allocated in RFC6598.
- Planning Your Network Topology
- BGP Configuration
- Notifying Google
- Changing Routing For Privately Addressed Users
- Testing And Diagnostics
For information about non-CGN GGC node-configuration, please see the Google Global Cache - Install Guide.
Planning Your Network Topology
The following figure shows a hypothetical network topology used as an example for this document.
Traffic between the users in the private IP space and the general Internet is NAT'ted. A route will be added to allow traffic between these users and the High Traffic IPs in the GGC sub-net to bypass the CGN.
Google will provide you with a list of High Traffic IPs for each GGC node.
Example CGN Topology
In the route illustration above:
- A user with IP 192.168.1.2 can reach a High Traffic IP (e.g. 208.117.227.36) on the GGC servers, bypassing your CGN. The GGC server sees the user's source IP as 192.168.1.2. It can send traffic back to 192.168.1.2, bypassing the CGN servers (the red path).
- A user with IP 192.168.1.2 can reach all GGC IPs other than the High Traffic IPs (e.g., 208.117.227.31) through the CGN. The GGC server on 208.117.227.31 sees the user's source IP NAT'ed as 208.117.8.0/24. It can to send traffic back to192.168.1.2 through the CGN servers (the green path).
- Traffic from the user 192.168.1.2 is directed towards sites on the public Internet through the CGN. The public Internet will see the user coming from a NAT'ed IP in 208.117.8.0/24 (the green path).
- Traffic between other users, in publicly routable space, and all GGC server IPs, is routed independently of the CGN. It is seen directly by the GGC servers (and any other server on the public Internet) with a public non-NAT'ed source address (the blue path).
BGP Configuration
BGP communities are used to identify which prefixes are private user IPs, and which prefixes contain public addresses of your CGN servers.
| Community | NAT Configuration Range |
|---|---|
|
15169:12000 |
Indicates a prefix contains a NAT device. |
|
15169:12100 |
Indicates a prefix is assigned to users who are NATed (it is either RFC1918, RFC6598, or usurped globally unique address space). |
In the example network, the advertisements would be:
BGP communities example
| CIDR | Usage | Community |
|---|---|---|
| 208.117.8.0/24 | CGN device | 15169:12000 |
| 192.168.0.0/16 | Users routed via CGN | 15169:12100 |
| 208.0.0.0/8 | Users not routed via CGN | none |
Google requires all GGC nodes in the same GGC Network Location (GNL) to have the same prefixes and community tags advertised to them.
A GNL is a set of GGC nodes serving the same users, with the same failover policy.
If you have multiple NAT devices in the same GNL, you must ensure the following:
- All the CGN device prefixes are advertised to all GGC nodes in the GNL, with community 15169:12000.
- All the private user addresses are advertised to all the GGC nodes in the GNL, with community 15169:12100.
- You must ensure that CGN bypass can be enabled for all nodes in the GNL, and for all user prefixes, all at once. This ensures that all the private IP addresses have a direct route to all the GGC nodes.
This means that you cannot reuse private addresses behind multiple NAT devices at the same GNL. You can reuse private addresses behind NAT devices at different GNLs.
You can configure these community tags now. Until Google verifies CGN support for the node, and you reconfigure your routing, users will continue to be served via the public IP addresses of your CGN servers.
Changes to your BGP configuration may take up to 2 hours to be seen in all Google systems.
For general details about configuring BGP for GGC, please see the Google Global Cache - Install Guide.
For more detail on use of community tags, please see BGP Community Support for Google Serving.
Notifying Google
Once you have planned your network topology and added community tags to your BGP advertisements, inform GGC Operations that the node is ready for CGN support by emailing ggc@google.com.
Include the following information:
- GGC node names, for all nodes in this GNL. Refer to the Assets page at the ISP Portal for a list.
- Confirmation that you have added community tags to your advertisements, at all nodes in the GNL.
- Confirmation that you are ready to bypass the CGN for connections to High Traffic IPs.
- Any special scheduling requirements.
Google will verify your prefix advertisements and the GGC node configurations.
We will then provide you with:
- Verification that we see appropriate prefix announcements and community tags.
- A list of High Traffic IPs on the GGC servers, for you to use in your routing configuration.
Changing Routing for Privately Addressed Users
Once Google has verified CGN support for the GGC nodes in this GNL, we will let you know that it's ready for CGN-bypassed traffic.
When we've done this, you can then re-configure your routing, for all nodes in the GNL.
You should ensure that all traffic between private user IPs tagged with 15169:12100, and the High Traffic IPs on the GGC servers bypass the CGN.
Let us know when you've made this change, and we'll verify that we see traffic directly from private user IPs.
Testing and Diagnostics
Playing a Test Video
Follow the procedure below to play a test video. If the system works as intended, the video traffic should be served from the GGC node, bypassing the CGN. This procedure is similar to the test procedure in the Google Global Cache - Install Guide.
The test below was made with Google Chrome version 7.9. running in Debian Linux. Output from other Chrome versions, or other Operating Systems, may look different.
Using Google Chrome from a PC in one of the tagged private user IP ranges:
- Open 'Developers Tools' (Menu Button > More Tools > Developer Tools or `Shift+Ctrl+I`).
- Open the 'Network' tab.
- Filter output by "videoplayback".
- Click "XHR".
- Play a video that is popular in your location. In the US, the https://www.youtube.com/feed/trending URL will help identify one. Your location may offer a different URL.
- Skip any ads. Ensure that the video has played properly for a few seconds.
- Click the "Pause" button in the video playback window.
- Click the round red button at the top left of the Developer Tools window to stop recording the network activity.
- Click the most recent "videoplayback" line.
- Expand the "Request Headers".
- Look for the ":authority:" line, which shows an obfuscated server name.
- In this test it was "r2---sn-5uaezn6k.googlevideo.com"
- Resolve the server name using "nslookup", "host", "dig", or similar tools. The results should look similar to this:
nslookup r4---sn-5ualdnl7.googlevideo.com
Server: <your server name>
Address: <your server ip>
Non-authoritative answer:
Name: r4.sn-5ualdnl7.googlevideo.com
Addresses: 2607:f8b0:4002::9
74.125.6.9
Aliases: r4---sn-5ualdnl7.googlevideo.com
If the resulting addresses (74.125.6.9 and 2607:f8b0:4002::9 in this example) are addresses in the sub-nets allocated to the GGC node, then video is playing correctly from the cache.
This image shows output from the Chrome Developer Tools while performing the video playback test described above.
The video playback test described above was made with Google Chrome version 7.9. running in Debian Linux. Output from other Chrome versions, or other Operating Systems, may look different.
Verifying your Public IP address
The following test should be made from a PC in one of the "15169:12100" tagged private user IP ranges.
The following URL gives information on how Google systems see your IP address. This will assist us in determining if CGN-bypass is working correctly.
https://redirector.googlevideo.com/report_mapping?di=no displays the IP address that you used to reach Google at AS15169, followed by => and some internal information only useful for our support teams.
The response should look like this. Your IP address is on the left.
208.117.8.1 => yourispname-abc1 (208.117.8.0/24)Verifying your Private IP address
You will need to specify one of the High traffic IPs from the list given to you by Google. In this example it is 208.117.227.36.
Adding /debug_info (https://208.117.227.36/debug_info) and navigating to the URL in your browser should display your private IP
The response should look something like:
client_ip = '192.168.1.2'
private_ip_ranges match: true
url_ip = "
destination_ip = '208.117.227.36'
date_time = '2013/10/10-21:26:15.424'
validation status code = SIGNATURE_MISMATCH
Check that the response includes a tagged private address for client_ip and has private_ip_ranges match: true