Carrier Grade NAT

Introduction

This document describes how to configure Google Global Cache (GGC) nodes to operate with your Carrier Grade NAT (CGN) systems. Once configured, this allows traffic between the GGC node and private user IPs to bypass the CGN.

GGC has no requirement that "private IP space" be in the RFC1918 allocation. Any non-internet-routed space in use in your network can bypass a CGN with this configuration. If appropriate for your usage, we recommend using the dedicated '100.64.0.0/10' CGN space allocated in RFC6598.

  1. Planning Your Network Topology
  2. BGP Configuration
  3. Notifying Google
  4. Changing Routing For Privately Addressed Users
  5. Testing And Diagnostics

For information about non-CGN GGC node-configuration, please see the Google Global Cache - Install Guide.

Planning Your Network Topology

The following figure shows a hypothetical network topology used as an example for this document.

Traffic between the users in the private IP space and the general Internet is NAT'ted. A route will be added to allow traffic between these users and the High Traffic IPs in the GGC sub-net to bypass the CGN.

Google will provide you with a list of High Traffic IPs for each GGC node.

Example CGN Topology

 

In the route illustration above:

  • A user with IP 192.168.1.2 can reach a High Traffic IP (e.g. 208.117.227.36) on the GGC servers, bypassing your CGN. The GGC server sees the user's source IP as 192.168.1.2. It can send traffic back to 192.168.1.2, bypassing the CGN servers (the red path).
  • A user with IP 192.168.1.2 can reach all GGC IPs other than the High Traffic IPs (e.g., 208.117.227.31) through the CGN. The GGC server on 208.117.227.31 sees the user's source IP NAT'ed as 208.117.8.0/24. It can to send traffic back to192.168.1.2 through the CGN servers (the green path).
  • Traffic from the user 192.168.1.2 is directed towards sites on the public Internet through the CGN. The public Internet will see the user coming from a NAT'ed IP in 208.117.8.0/24 (the green path).
  • Traffic between other users, in publicly routable space, and all GGC server IPs, is routed independently of the CGN. It is seen directly by the GGC servers (and any other server on the public Internet) with a public non-NAT'ed source address (the blue path).
Do not configure private user IP addresses to bypass the CGN until you have received confirmation from Google that the GGC nodes are ready for CGN-bypassed traffic. Changing your routing without prior approval is likely to lead to very poor performance for users. It may even cause complete failure for some user requests.

BGP Configuration

BGP communities are used to identify which prefixes are private user IPs, and which prefixes contain public addresses of your CGN servers.

Community NAT Configuration Range

15169:12000

Indicates a prefix contains a NAT device.

15169:12100

Indicates a prefix is assigned to users who are NATed (it

is either RFC1918, RFC6598, or usurped globally unique

address space).

 

 

 

 

 

 

 

 

In the example network, the advertisements would be:

BGP communities example

CIDR Usage Community
208.117.8.0/24 CGN device 15169:12000
192.168.0.0/16 Users routed via CGN 15169:12100
208.0.0.0/8 Users not routed via CGN none

 

 

 

 

 

 

Google requires all GGC nodes in the same GGC Network Location (GNL) to have the same prefixes and community tags advertised to them.

A GNL is a set of GGC nodes serving the same users, with the same failover policy.

If you have multiple NAT devices in the same GNL, you must ensure the following:

  • All the CGN device prefixes are advertised to all GGC nodes in the GNL, with community 15169:12000.
  • All the private user addresses are advertised to all the GGC nodes in the GNL, with community 15169:12100.
  • You must ensure that CGN bypass can be enabled for all nodes in the GNL, and for all user prefixes, all at once. This ensures that all the private IP addresses have a direct route to all the GGC nodes.

This means that you cannot reuse private addresses behind multiple NAT devices at the same GNL. You can reuse private addresses behind NAT devices at different GNLs.

You can configure these community tags now. Until Google verifies CGN support for the node, and you reconfigure your routing, users will continue to be served via the public IP addresses of your CGN servers.

Changes to your BGP configuration may take up to 2 hours to be seen in all Google systems.

For general details about configuring BGP for GGC, please see the Google Global Cache - Install Guide.

For more detail on use of community tags, please see BGP Community Support for Google Serving.

Notifying Google

Once you have planned your network topology and added community tags to your BGP advertisements, inform GGC Operations that the node is ready for CGN support by emailing ggc@google.com.

Include the following information:

  • GGC node names, for all nodes in this GNL. Refer to the Assets page at the ISP Portal for a list.
  • Confirmation that you have added community tags to your advertisements, at all nodes in the GNL.
  • Confirmation that you are ready to bypass the CGN for connections to High Traffic IPs.
  • Any special scheduling requirements.

Google will verify your prefix advertisements and the GGC node configurations.

We will then provide you with:

  • Verification that we see appropriate prefix announcements and community tags.
  • A list of High Traffic IPs on the GGC servers, for you to use in your routing configuration.

Changing Routing for Privately Addressed Users

Once Google has verified CGN support for the GGC nodes in this GNL, we will let you know that it's ready for CGN-bypassed traffic.

When we've done this, you can then re-configure your routing, for all nodes in the GNL.

You should ensure that all traffic between private user IPs tagged with 15169:12100, and the High Traffic IPs on the GGC servers bypass the CGN.

Let us know when you've made this change, and we'll verify that we see traffic directly from private user IPs.

Testing and Diagnostics

Playing a Test Video

Follow the procedure below to play a test video. If the system works as intended, the video traffic should be served from the GGC node, bypassing the CGN. This procedure is similar to the test procedure in the Google Global Cache - Install Guide.

The test below was made with Google Chrome version 7.9. running in Debian Linux. Output from other Chrome versions, or other Operating Systems, may look different.

Using Google Chrome from a PC in one of the tagged private user IP ranges:

  1. Open 'Developers Tools' (Menu Button > More Tools > Developer Tools or `Shift+Ctrl+I`).
  2. Open the 'Network' tab.
  3. Filter output by "videoplayback".
  4. Click "XHR".
  5. Play a video that is popular in your location. In the US, the https://www.youtube.com/feed/trending URL will help identify one. Your location may offer a different URL.
  6. Skip any ads. Ensure that the video has played properly for a few seconds.
  7. Click the "Pause" button in the video playback window.
  8. Click the round red button at the top left of the Developer Tools window to stop recording the network activity.
  9. Click the most recent "videoplayback" line.
  10. Expand the "Request Headers".
  11. Look for the ":authority:" line, which shows an obfuscated server name.
  12. In this test it was "r2---sn-5uaezn6k.googlevideo.com"
  13. Resolve the server name using "nslookup", "host", "dig", or similar tools. The results should look similar to this:

nslookup r4---sn-5ualdnl7.googlevideo.com
Server:  <your server name>
Address:  <your server ip>

Non-authoritative answer:
Name:    r4.sn-5ualdnl7.googlevideo.com
Addresses:  2607:f8b0:4002::9
          74.125.6.9
Aliases:  r4---sn-5ualdnl7.googlevideo.com

If the resulting addresses (74.125.6.9 and 2607:f8b0:4002::9 in this example) are addresses in the sub-nets allocated to the GGC node, then video is playing correctly from the cache.

This image shows output from the Chrome Developer Tools while performing the video playback test described above.

The video playback test described above was made with Google Chrome version 7.9. running in Debian Linux. Output from other Chrome versions, or other Operating Systems, may look different.

The base web pages of www.youtube.com may not be served from the cache. These host names will typically not resolve to the GGC node, and will not bypass the CGN.

Verifying your Public IP address

The following test should be made from a PC in one of the "15169:12100" tagged private user IP ranges.

The following URL gives information on how Google systems see your IP address. This will assist us in determining if CGN-bypass is working correctly.

https://redirector.googlevideo.com/report_mapping?di=no displays the public address of your CGN servers, and the GGC node name.

The response should look like this:

208.117.8.1 => yourispname-abc1 (208.117.8.0/24)

Verifying your Private IP address

You will need to specify one of the High traffic IPs from the list given to you by Google. In this example it is 208.117.227.36.

Adding /debug_info (https://208.117.227.36/debug_info) and navigating to the URL in your browser should display your private IP

The response should look something like:

client_ip = '192.168.1.2'

       private_ip_ranges match: true

url_ip = "

destination_ip = '208.117.227.36'

date_time = '2013/10/10-21:26:15.424'

validation status code = SIGNATURE_MISMATCH

Check that the response includes a tagged private address for client_ip and has private_ip_ranges match: true

Was this helpful?
How can we improve it?
Search
Clear search
Close search
Google apps
Main menu
Search Help Center
false
false