Step 1: Locate the uncrawlable URL
Uncrawlable pages can affect your Dynamic Search Ads in 3 ways:
- If you select
URL-Equalsas your targeting type and your URL is uncrawlable, you'll see the status “Disapproved: Page cannot be crawled” on your dynamic ad target.
- If you use page feeds and a URL in your feed is uncrawlable, you'll see the status “Disapproved: Page cannot be crawled” on the row for that URL in the shared library.
- If you use any other targeting mechanism and a URL isn't or cannot be crawled, Google Ads won’t know about it, so no error message will be displayed. If a URL you’re attempting to target isn't getting traffic, you can troubleshoot by first targeting it through
URL-Equalsor page feeds. The URL will either get traffic or you'll get an error explaining why it isn’t getting traffic.
Step 2: Make sure that Google AdsBot can crawl the landing page
- Copy the URL and paste it in a web browser's address bar.
/robots.txtto the end of the domain name, then press Enter. For example, if your landing page is
http://www.example.com/folder1/folder2/, check what robots are allowed to crawl the domain by visiting the URL:
- Look for the line
- If it's not there, work with the webmaster to add it.
- If the page is blank, work with the webmaster to add a
robots.txtfile to the domain.
User-agent: AdsBot-Googleis there, make sure it isn't followed by a line that says
Disallow: /. Also make sure AdsBot-Google isn't restricted by another line in the file. About
robots.txtfiles and Google crawlers
- If the landing page is a subdomain, check if the subdomain has a separate
robots.txtfile. For example,
https://subdomain.domain.com/robots.txt. Make sure this
- Open the source of the URL. In Chrome, you can do this by opening
view-source:THE_URL. For example, if your landing page is
http://www.example.com/folder1/folder2/, you can check what robots are allowed to crawl the domain by navigating to
- Look for the string “
- If it’s not there, this is not the source of the issue.
- If the string is within a tag like
<meta name="AdsBot-Google" content="noindex" />, work with your webmaster to remove it. Learn more about meta tags that Google understands
Other reasons that a landing page can't be crawled
If the Google AdsBot is allowed to crawl the landing page (
User-agent: AdsBot-Google is already included in the site's
robots.txt file), there may be another reason the page can't be crawled:
- The landing page isn't loading. If you open the page with a web browser and see an error (for example, 404 Not Found, Site not reachable, Your session has expired), contact the site's webmaster to make sure the landing page loads in any location, environment (mobile and desktop), or browser.
- The landing page requires sign-in. All ad landing pages must be publicly accessible. Use another landing page or work with the webmaster to remove the sign-in requirement from the page.
- The landing page has too many forwards or redirects. Landing pages must have fewer than 10 redirects. Work with webmaster to reduce the number of redirects. Learn more about web forwarding