Use Fetch as Google

Test if Google can crawl your web page

The Fetch as Google tool enables you to test how Google crawls or renders a URL on your site. You can use Fetch as Google to see whether Googlebot can access a page on your site, how it renders the page, and whether any page resources (such as images or scripts) are blocked to Googlebot. This tool simulates a crawl and render execution as done in Google's normal crawling and rendering process, and is useful for debugging crawl issues on your site.

Open Fetch as Google

For mobile apps, use the equivalent Fetch as Google for Apps tool

Using the tool

  1. In the textbox, enter the path component of a URL on your site that you want Googlebot to fetch, relative to the site root. Leaving the textbox blank fetches the site root page. For example, if the current property is "http://example.com", specifying "stores/indiana/1234.html" would fetch http://example.com/stores/indiana/1234.html
    Fetch restrictions:
    • Fetched URLs are limited to the current site: for example, if logged in to http://example.com you cannot fetch a URL from https://example.com or http://m.example.com.
    • The fetch does not send any cookies, login information, or other state information.
    • The fetch will not follow a redirect. If you fetch a page with a redirect, you will have to follow it manually as described in the "Redirected" fetch status description below.
  2. Optionally choose a type of Googlebot you wish to perform the fetch as. This affects the crawler making the fetch, and also the rendering for a Fetch and Render request. The following types are available:
    1. Desktop [Default]
    2. Mobile: Smartphone
    3. Mobile: cHTML (a subset of mostly Japanese feature phones). Rendering not supported.
    4. Mobile: XHTML/WML (feature phones). Rendering not supported
  3. Click either Fetch or Fetch and Render:
    • Fetch: Fetches a specified URL in your site and displays the HTTP response. Do not request or run any associated resources (such as images or scripts) on the page. This is a relatively quick operation that you can use to check or debug suspected network connectivity or security issues with your site, and see the success or failure of the request.
    • Fetch and render: Fetches a specified URL in your site, displays the HTTP response and also renders the page according to a specified platform (desktop or smartphone). This operation requests and runs all resources on the page (such as images and scripts). Use this to detect visual differences between how Googlebot sees your page and how a user sees your page.
  4. The request will be added to the fetch history table, with a "pending" status. When the request is complete, the row will show the success or failure of the request and some basic information. Click any non-failed fetch row in the table to get additional details about the request, including HTTP response headers and data, blocked resources, and (for Fetch and Render requests) a view of the rendered page.
  5. If the request is less than four hours old, you can tell Google to re-crawl and possibly re-index the fetched page, and optionally any direct links on your site.

You have a weekly quota of 500 fetches. When you are approaching your limit, you will see a notification on the report.

Analyzing your Fetch results

The fetch history table shows the last 100 fetch requests. To see details for a completed fetch, click on the corresponding row in the fetch history table. The following statuses can be returned in response to a fetch:

Fetch Statuses

Status

Fetch

Notes & Next Steps

Complete

Google successfully contacted your site and crawled your page.

On the Fetch as Google page, click the table row that corresponds to your fetch to see more details about the fetch results.

Redirected

The server responded with a redirect. The Fetch as Google tool does not follow redirects.

Although the actual Google crawler follows redirects, the Fetch as Google tool will not. You must follow a redirect manually:

  • If the redirect is to the same property, the tool displays a button that allows to quickly follow the redirect by populating the fetch box with the redirect URL.
  • If the URL redirects to another property that you own, you can click "Follow" to autopopulate the URL box, then copy the URL, switch views to the new site, and then paste the URL into the fetch box.

You can inspect the HTTP response on the fetch details page to see the redirect details. Locate the HTTP error code to learn more.

Not found

The resource could not be found (404 or 410 HTTP response codes).

This error indicates that you might see the HTTP 404 error code when you access your page using a web browser.

Not authorized

Googlebot isn't authorized to access the resource (for example, a resource that requires a username and password).

This error indicates that you might see the HTTP 403 error code when you access your page using a web browser.

DNS not found

Google couldn’t retrieve the resource because the domain name wasn’t found.

Make sure you typed in your domain name (for example, www.example.com) properly so that Google can find your site server.

Blocked

The resource's host is blocking access to Googlebot by means of a robots.txt file.

This error is a common problem that you can fix by updating your robots.txt file.

Unreachable robots.txt

Googlebot can’t reach the resource host's robots.txt file. When that happens, Google avoids loading any resources from that host.

To resolve this issue, read our Help Center articles on how to create and test robots.txt files.

Unreachable

The resource host either took too long to reply or refused the request.

Check to see that your server is up and running.

Temporarily unreachable

1) Fetch as Google can’t currently fetch your URL because the server took too long to reply.

OR

2) Fetch as Google cancelled your fetch because too many consecutive requests were made to the server for different URLs.

Note the URL is not unreachable for all of Google-- it is just unreachable for the Fetch as Google simulation tool.

Error

An unspecified error prevented Google from completing the fetch.

If this error happens again, we ask that you contact Search Console product support.

 
Fetch and Render Statuses

Status

Fetch and Render

Notes & Next Steps

Complete

Google can successfully contact your site, crawl your page, and get every resource referenced by that page.

On the Fetch as Google page, click the on the table row that corresponds to your fetch to see more details about the results of the operation.

Partial

Google got a response from your site and fetched URL, but we could not reach some of the resources on that URL for rendering.

You can assess the gravity of the situation by clicking through the missing resources. A fetch with the Partial status could mean a small problem with the page (e.g. few or insignificant resources could not be retrieved) or a more serious one (e.g. many or important resources which include a large part of the page content are missing).

Note that occasionally, blocked resources reside on other domains, which you might not be able to control. If important page content is missing due a blocked resource on another site, you might want to contact the resource owner to allow Google to access that resource. It is also possible that the rendered URL references some resources that simply do not exist.

Redirected

1) At the fetching stage, when Google contacted the host server for your URL, the server told Google to visit a different URL.

OR

2) At the rendering stage, HTML meta tags and Javascript code might tell the Google to visit other sites.

Although the actual Google crawler follows redirects, the Fetch as Google tool will not. You must follow a redirect manually:

  • If the redirect is to the same property, the tool displays a button that allows to quickly follow the redirect by populating the fetch box with the redirect URL.
  • If the URL redirects to another property that you own, you can click "Follow" to autopopulate the URL box, then copy the URL, switch views to the new site, and then paste the URL into the fetch box.

You can inspect the HTTP response on the fetch details page to see the redirect details. Locate the HTTP error code to learn more.

You can also search through your meta tags and Javascript code if the redirects are happening at the rendering level.

Not found

The resource could not be found (404 or 410 HTTP response codes).

This error indicates that, when you access your page using a web browser, you might see the HTTP 404 error code.

Not authorized

Googlebot isn't authorized to access the resource (for example, a resource that requires a username and password).

This error indicates that, when you access your page using a web browser, you might see the HTTP 403 error code.

DNS not found

Google couldn’t retrieve the resource because the domain name wasn’t found.

Make sure you typed in your domain name (for example, www.example.com) properly so that Google can find your site server.

Blocked

The resource's host is blocking access to Googlebot by means of a robots.txt file.

This is a common problem that you can fix by updating your robots.txt file.

If your site address is at the root of a domain (for example, www.example.com and not www.example.com/my_site/), you can use the robots.txt Tester tool to diagnose why the URL is blocked from Google.

Unreachable robots.txt

Googlebot can’t reach the resource host's robots.txt file. When that happens, Google avoids loading any resources from that host.

Learn more about how to create and test robots.txt files.

Unreachable

The resource host either took too long to reply or refused the request.

Check to see that your server is up and running.

Temporarily unreachable

1) Fetch as Google can’t currently fetch your URL because the server took too long to reply.

OR

2) Fetch as Google cancelled your fetch because too many consecutive requests were made to the server for different URLs.

Note the URL is not unreachable for all of Google: it is just unreachable for the Fetch as Google simulation tool.

Error

An unspecified error prevented Google from completing the fetch.

If this happens again, we ask that you contact Search Console product support.

Was this article helpful?