Search
Clear search
Close search
Google apps
Main menu

Index Coverage Status report

New Search Console Beta experience

About this report

Use this report to learn which of your pages have been indexed, and how to fix pages that could not be indexed. Each bar in the graph represents the total number of URLs in a particular status (valid, error, and so on) as known by Google.

Classic Search Console Users: Classic Search Console reported many stale URLs that no longer appear in Search results. Therefore, if you used the Index Status report in classic Search Console, you may notice that the total indexed numbers shown in the new Index Status report are lower than the total indexed numbers in the older Index Status report. This is because the new report focuses on pages that are actually seen in Google Search in order to provide more useful and fresher data.

What to look for

Ideally you should see a gradually increasing count of valid indexed pages as your site grows.

  • If you see a spike in indexing errors, this might be caused by a change in your template that introduces a new error, a change in robots.txt blocking or site access settings, or a change in your sitemap that includes many pages that can't be indexed for some reason.
  • If you see a drop in total indexed pages without corresponding errors, this might mean that you are blocking access to your existing pages (via robots.txt,  'noindex', or requiring auth). If that is not the issue, look at the warning or informational/excluded messages, sorted by number of pages affected, to see what might be causing this drop.
  • If you have a significant number of pages that are not indexed and you think they should be, look at warning and informational/excluded URLs for clues. You might be roboting or noindexing many of your pages. Look for these or other warning or informational/excluded URLs affecting your pages, sorted by number of pages affected.
How are these URLs found? Google discovers URLs through many means, most commonly by following links from crawled pages or by sitemaps. Sometimes these links are wrong (and can lead to 404s on your site). Sometimes the page existed, and has disappeared. But once Google has heard of a URL it will continue to try to crawl it for a while. That's natural; if you want to prevent that, you can block indexing, block access, or use a 301 redirect (where appropriate).

Top level report

The top level report shows the index status of all pages that Google has attempted to crawl on your site, grouped by status and reason.

Status

Each page can have one of the following general status classes:

  • Valid: The page was indexed.
  • Error: The page has not been indexed. See the specific error type description below to learn more, and how to fix the error. You should concentrate on these issues first.
  • Warning: Page is indexed, or was until recently, and has an issue that you should be aware of.
  • Informational/Excluded: The page is not included in the index for reasons that you typically cannot affect. The page might be in an intermediate stage of the indexing process, or is behaving as expected.

Status type

Each status (error, warning, informational/excluded, valid) can have a specific type. See Status type descriptions below for a description of each status type, and how to handle it.

URL discovery dropdown filter

The dropdown filter above the chart enables you to filter index results by the mechanism through which Google discovered the URL. The following values are available:

  • All known pages [Default] - Show all URLs discovered by Google through any means.
  • All submitted pages - Show only pages submitted in a sitemap using Search Console (not a sitemap discovered from a robots.txt file or a sitemap ping).
  • Specific sitemap URL - Show only URLs listed in a specific sitemap that was submitted using Search Console. If it is a sitemap index, all URLs in any included sitemaps are reported.

A URL is considered submitted by a sitemap even if it was actually discovered through some other mechanism (for example, by organic crawling from another page).

Drilldown report by status and reason

Clicking on a row in the top page will show details for a specific status type. The reason report contains the following information:

  • A graph showing URLs by general status (valid, error, warning, informational/excluded).
  • A table showing the URLs by status type, and the last time that URL was crawled.

Important: Seeing a URL marked with an issue that you've already fixed? Perhaps you fixed the issue AFTER the last Google crawl. Therefore, if you see a URL with an issue that you have fixed, be sure to check the crawl date for that URL:

  • If the URL was recrawled after your fix, we couldn't confirm your fix. Check and confirm your fix and wait for a recrawl.
  • If the URL was crawled before the fix, either wait for Google to recrawl the page, or click "start fixing" (if displayed) and fix the issue using the issue management flow.

Troubleshooting your pages

  1. Filter out the warning and informational/excluded URLs on the top level report.
  2. See if you can find any correspondence between the total number of indexing errors or total indexed count and the sparkline for a specific error as a clue to which issue might be affecting your total error or total indexed page count.
  3. Fix error rows:
    1. If there is an increase in errors, look for frequency spikes in the row that happened at the same time as any error spikes in the top chart, and click the row to learn more in the drilldown report (described next).
    2. Click an error row to get to the drilldown page with more information (see below). Read the description about the specific error type to learn how to handle it best.
    3. Check back after a few days to see whether your error count has gone down.
  4. Remove the filter for warnings, and fix any warnings that indicate trouble, according to the issue description.
  5. Periodically remove the filter for informational/excluded URLs, sort them by number of affected pages, and scan them for any unwanted issues.

Status reasons

Here are the possible reasons for each of your pages.

Submitted vs not submitted

Any time you see an index result that uses the word "Submitted", it means that you have explicitly asked Google to index the URL by submitting it in a sitemap.

Valid


Pages with a valid status have been indexed.

Submitted and indexed: You submitted the URL for indexing, and it was indexed.

Indexed, not submitted in sitemap: The URL was discovered by Google and indexed. We recommend submitting all important URLs using a sitemap.

Indexed; consider marking as canonical: The URL was indexed. Because it has duplicate URLs, we recommend explicitly marking this URL as canonical.

Indexed, low interest: The page was indexed, but appears very rarely in Search results. It might be omitted from other Search Console reports.

Warning


Pages with a warning status might require your attention, and may or may not have been indexed, according to the specific result.


Indexed, though blocked by robots.txt: The page was indexed, despite being blocked by robots.txt. Robots.txt is not the correct mechanism to avoid being indexed. To avoid being indexed you should either use 'noindex' or prohibit anonymous access to the page using auth. You can use the robots.txt tester to determine which rule is blocking this page. Because of the robots.txt, any snippet shown for the page will probably be sub-optimal.

Indexed URL now blocked by robots.txt: In our most recent index attempt we discovered that this page is now blocked by robots.txt. The page was previously indexed, but probably will be dropped and not updated or indexed again until you unblock the page. Use the robots.txt tester to determine which rule is blocking this page.

Indexed URL now marked ‘noindex’: In our most recent index attempt we discovered that this page is now marked as noindex. The page has therefore been dropped from our index. If you want this page to be added back to the index you should remove that directive from your page.

Indexed URL now seems to be a Soft 404: A page previously indexed now returns a page that seems to be a soft 404 page. This means it returns a user-friendly "not found" message, but does not return a 404 error code. We recommend returning a 404 error code to ensure proper indexing of the page. The earlier cached version of the page might still be available through Search results. If you want to remove it from our cache, use the Remove URLs tool and specify removal from the cache as well as from Search results.

Indexed URL now returns access denied: A page previously indexed now returns an "access denied" error (401/403). The page still exists in the index and can be returned as a search result; if you want to remove the previously indexed page, use the Remove URLs tool and specify removal from the cache as well as from Search results.

Indexed URL now not found (404): A page previously indexed now returns a not found error (404). The earlier indexed version will not be returned as the main link in a Search result, but the page still exists in the index and can be returned as a search result; if you want to remove the previously indexed page, use the Remove URLs tool and specify removal from the cache as well as from Search results.

Error


Pages with errors have not been indexed.


Server error (5xx): Your server returned a 500-level error when the page was requested.

Redirect error: The URL was a redirect error. Could be one of the following types: it was a redirect chain that was too long; it was a redirect loop; the redirect URL eventually exceeded the max URL length; there was a bad or empty URL in the redirect chain.

Submitted URL has crawl issue: You submitted this page for indexing, and Google encountered an unspecified crawling error that doesn't fall into any of the other categories. Try debugging your page using Fetch as Google.

Submitted URL blocked by robots.txt: You submitted this page for indexing, but the page is blocked by robots.txt. Try testing your page using the robots.txt tester.

Submitted URL marked ‘noindex’: You submitted this page for indexing, but the page has a 'noindex' directive either in a meta tag or HTML response. If you want this page to be indexed, you must remove the tag or HTTP response.

Submitted URL seems to be a Soft 404: You submitted this page for indexing, but the server returned what seems to be a soft 404.

Submitted URL returns unauthorized request (401): You submitted this page for indexing, but Google got a 401 (not authorized) response. Either remove authorization requirements for this page, or else allow Googlebot to access your pages by verifying its identity.

Submitted URL not found (404): You submitted a non-existent URL for indexing.

Submitted URL removed by user request: You submitted this page for indexing, but the page has a standing request to be removed from the index. Removal requests are only good for a specified period of time (see the linked documentation). After that period, Googlebot may go back and index the page, even if you do not submit another index request. If you do not want the page to be indexed, use 'noindex', require authorization for the page, or remove the page. If you are a verified site owner, you can use the URL removals tool to see who submitted a URL removal request, or to cancel the URL removal request.

Informational/Excluded


These pages are typically not indexed, but probably for intentional reasons.


Blocked by ‘noindex’ tag: When Google tried to index the page it encountered a 'noindex' directive, and therefore did not index it. If you do not want the page indexed, you have done so correctly. If you do want this page to be indexed, you should remove that 'noindex' directive.

Blocked by page removal tool: The page is currently blocked by a URL removal request. Removal requests are only good for a specified period of time (see the linked documentation). After that period, Googlebot may go back and index the page, even if you do not submit another index request. If you do not want the page to be indexed, use 'noindex', require authorization for the page, or remove the page. If you are a verified site owner, you can use the URL removals tool to see who submitted a URL removal request.

Blocked by robots.txt: This page was blocked to Googlebot with a robots.txt file. You can verify this using the robots.txt tester. Note that this does not mean that the page won't be indexed through some other means. If Google can find other information about this page without loading it, the page could still be indexed (though this is less common). To ensure that a page is not indexed by Google, remove the robots.txt block and use a 'noindex' directive.

Blocked due to unauthorized request (401): The page was blocked to Googlebot by a request for authorization (401 response). If you do want Googlebot to be able to crawl this page, either remove authorization requirements, or allow Googlebot to access your pages by verifying its identity.

Crawl anomaly: An unspecified anomaly occurred when fetching this URL. This could mean a 4xx- or 5xx-level response code; try fetching the page using Fetch as Google to see if it encounters any fetch issues. The page was not indexed.

Crawled: The page was crawled by Google, but not indexed. It may or may not be indexed in the future; no need to resubmit this URL for crawling.

Discovered - currently not indexed: The page was found by Google, but not crawled yet.

Alternate page with proper canonical tag: This page is a duplicate of a page that Google recognizes as canonical, and it correctly points to that canonical page, so nothing for you to do here!

Duplicate page without canonical tag: This page has duplicates, none of which is marked canonical. We think this page is not the canonical one. You should explicitly mark the canonical for this page. To learn which page is the canonical, click the table row to run an info: query for this URL, which should list its canonical page.

Duplicate non-HTML page: A non-HTML page (for example, a PDF file) is a duplicate of another page that Google has marked as canonical. Typically only the canonical URL will be shown in Google Search. If you like, you can specify a canonical page using the Link HTTP header in a response.

Google chose different canonical than user: This URL is marked as canonical for a set of pages, but Google thinks another URL makes a better canonical. Because we consider this page a duplicate, we did not index it; only the canonical page is indexed. We recommend that you explicitly mark this page as a duplicate of the canonical URL. To learn which page is the canonical, click the table row to run an info: query for this URL, which should list its canonical page.

Not found (404): This page returned a 404 error when requested. The URL was discovered by Google without any explicit request to be crawled. Google could have learned of the URL through different ways: for example, another page links to it, or it existed previously and was deleted. Googlebot will probably continue to try this URL for some period of time; there is no way to tell Googlebot to permanently forget a URL, although it will crawl it less and less often. 404 responses are not a problem, if intentional. If your page has moved, use a 301 redirect to the new location. Read here to learn more about how to think about 404 errors on your site.

Page removed because of legal complaint: The page was removed from the index because of a legal complaint.

Page with redirect: The URL is a redirect, and therefore was not added to the index.

Queued for crawling: The page is in the crawling queue; check back in a few days to see if it has been crawled.

Soft 404: The page request returns what we think is a soft 404 response. This means that it returns a user-friendly "not found" message without a corresponding 404 response code. We recommend returning a 404 response code for "not found" pages to prevent indexing of the page.

Submitted URL dropped: You submitted this page for indexing, but it was dropped from the index for an unspecified reason.

Submitted URL not selected as canonical: The URL is one of a set of duplicate URLs without an explicitly marked canonical page. You explicitly asked this URL to be indexed, but because it is a duplicate, and Google thinks that another URL is a better candidate for canonical, Google did not index this URL. Instead, we indexed the canonical that we selected. The difference between this status and "Google chose different canonical than user" is that, in this case, you explicitly requested indexing.

Was this article helpful?
How can we improve it?