Search
Clear search
Close search
Google apps
Main menu

The search appliance is unable to crawl URLs

Summary: In certain circumstances you expect the search appliance to crawl content, but the content does not show in search results.


Troubleshooting Steps:

  1. Check if the URL shows in the Index > Diagnostics > Index Diagnostics page in the Admin Console.
  2. If the URL does not show in the Index > Diagnostics > Index Diagnostics page in the Admin Console, verify:
    • The search appliance is configured to properly crawl the URL. See the Configuring a Crawl document for more info. For example, ensure that the document URL matches one of the patterns in the Follow Patterns. It must not match any of the patterns in the Do Not Follow Patterns. Use the pattern-testing utility ("Test these patterns" button) in the Admin Console to test the URL against these fields.
    • The URL is crawlable by the search appliance. See the What Content is not Crawled document for more info.
    • The search appliance has not exceeded its license limit. See the How Many URLs Can Be Crawled doc for more info.
  3. If the URL shows in the Index > Diagnostics > Index Diagnostics page and is in an error state, see the Errors from Web Servers document for the best way to troubleshoot the particular error.
  4. If the URL is not in an error state, but is not returned in search results, use the serving issue troubleshooting doc to further diagnose.
Was this article helpful?
How can we improve it?