Restricting content

To block certain articles or sections of your site from appearing in Google News, you'll have to restrict where our crawlers, or user-agents, can go. Google News uses the same user-agent as Web Search, the Googlebot. Googlebot obeys robots.txt files and robots meta tags for Google News. If you want to prevent Googlebot from indexing your site, you can create a robots.txt file. Note that this file needs to be placed at the root of your host or it might be ignored. For example, Googlebot would likely:

  • Follow: http://example.com/robots.txt
  • Ignore: http://example.com/site/robots.txt

Additionally, you can prevent the indexing of your site by using robots meta tags. For example, marking [ no follow ], would prevent us from extracting your articles.

For further information about robots.txt files and robots meta tags, please visit our Webmaster Help Center.

Keep in mind that you can block Google News from accessing your site while allowing access to Google Web Search. If you'd like to enable this option, you'll have to be very specific when creating your robots.txt file or meta tags. For more details, please visit our page on robots, meta tags, and HTML header specifications.