Notification

In this help center, you can find content for both Merchant Center Next and the classic Merchant Center experience. Look for the logo at the top of each article to make sure you're using the article for the Merchant Center version that applies to you. 

How to fix: Image not crawlable due to robots.txt

Update your robots.txt file to allow user-agents "Googlebot" and "Googlebot-Image" to crawl your site

A custom icon for Merchant Center Classic Article Header.

For one or more of your products, you’ve submitted an image (via the image link [image link] attribute) that cannot be crawled by Google because robots.txt forbids Google's crawler to download the image. Since images are a key part of the buying decision of online customers, Merchant Center products require an image. Your affected products will remain disapproved until we are able to crawl their images.

Please update the robots.txt file on your web server to allow Google's crawler to fetch the provided images. The robots.txt file can usually be found in the root directory of the web server (for instance, http://www.example.com/robots.txt). In order for us to access your whole site, ensure that your robots.txt file allows both user-agents Googlebot-image (used for images) and Googlebot (used for web pages) to crawl your site. You can do this by changing your robots.txt file as follows:

User-agent: Googlebot
Disallow:
User-agent: Googlebot-image
Disallow:

Learn more about how to configure robots.txt

You can test your current configuration with the URL inspection tool.

If you have fixed these issues and updated your products via a new feed upload or the Content API, the errors you see should disappear within a couple of days. This time allows us to verify that we can crawl the images that are provided, after which they will start showing up in ads and free listings. If you want to speed up the process you can increase Google's crawl rate.

Was this helpful?

How can we improve it?

Need more help?

Try these next steps:

false
Search
Clear search
Close search
Google apps
Main menu
205216172489790897
true
Search Help Center
true
true
true
true
true
71525