/webmasters/community?hl=en
/webmasters/community?hl=en
8/7/09
Original Poster
emetic

Why would Sitemap only index one URL of 51 total that webmaster tools found and how can you tell which one it indexed?

Here's the error message that I got:
We encountered an error while trying to access your Sitemap. Please ensure your Sitemap follows our guidelines and can be accessed at the location you provided and then resubmit.
 
On one hand it seems to be having trouble accessing the sitemap as the error message implies but on the other hand it did seem to find the sitemap since it reported the 51 links in the sitemap.
 
Here's some additional detail: On my Site www.engineeringfirms.com I have the same Spry Menu on each page for ease of navigation through the site. The menu has the same links defined as you would expect for this navigation scheme. Would that have any effect on how many of the 51 links found in the sitemap got indexed?
 
My site is a simple database driven site that should be a no brainer for creating a simple sitemap and indexing all the simple links that guide a person from one database to the next. If I made some mistake that is keeping all my 51 links the webmaster tools found in my sitemap, please let me know. Thanks.2009-08-07
 
Should be: If I made some mistake that is keeping all my 51 links the webmaster tools found in my sitemap from being indexed, please let me know. Thanks.
Community content may not be verified or up-to-date. Learn more.
Recommended Answer
Was this answer helpful?
How can we improve it?
All Replies (6)
John
8/7/09
John
Google hasn't seen the links yet, the last cache of the homepage is nothing but a under construction page:
http://74.125.95.132/search?sourceid=navclient-ff&ie=UTF-8&q=cache%3Ahttp%3A%2F%2Fwww.engineeringfirms.com%2F


Having a sitemap does not promise crawling, indexing, and/or ranking.

Per these Google sources:

1) "We cannot make any predictions or guarantees about when or if your URLs will be crawled or added to our index. Over time, we expect both coverage and time-to-index to improve as we refine our processes and better understand webmasters' needs.
"
http://www.google.com/support/webmasters/bin/answer.py?answer=34575

2) "Sitemaps provide an extra way for us to understand what pages you have on your site, and can help speed up the discovery of new and updated pages on your site. But neither HTML nor XML Sitemaps replace the normal crawling process. Having a Sitemap will not increase your ranking in Google's search results pages—but it will not reduce it, either."
http://sites.google.com/site/webmasterhelpforum/en/faq--sitemaps-malware--other-issues/#sitemap-penalty

3) "We don't guarantee that we'll crawl or index all of your URLs. For example, we won't crawl or index image URLs contained in your Sitemap. However, we use the data in your Sitemap to learn about your site's structure, which will allow us to improve our crawler schedule and do a better job crawling your site in the future. In most cases, webmasters will benefit from Sitemap submission, and in no case will you be penalized for it."
http://www.google.com/support/webmasters/bin/answer.py?answer=35184

4) "Please note that submitting a Sitemap doesn't guarantee that all pages of your site will be crawled or included in our search results."
http://www.google.com/support/webmasters/bin/answer.py?answer=34397

From the same article:

Although Google crawls billions of pages, it's inevitable that some sites will be missed. When our spiders miss a site, it's frequently for one of the following reasons:

-The site isn't well connected through multiple links from other sites on the web.
-The site launched after Google's most recent crawl was completed.
-The design of the site makes it difficult for Google to effectively crawl its content.
-The site was temporarily unavailable when we tried to crawl it or we received an error when we tried to crawl it. You can use Google Webmaster Tools to see if we recieved errors (http://www.google.com/support/webmasters/support/webmasters/bin/answer.py?answer=35120) when trying to crawl your site.


A site that is not well connected through multiple likes from other sites is much less likely to be indexed.



Dnyhagen
8/7/09
Dnyhagen
@emetic . . . Bingo! John's right . . . as usual.  I was looking at the trees and failed to notice the forest.
8/7/09
Original Poster
emetic
Dnyhagen and John,
 
Ok, I know that Google has not been back to crawl my site since I removed the "Under Consturction" page and went live with the site a couple of weeks ago. So are you saying that when the results of submitting my sitemap through Webmaster tools showed 1 URL indexed it meant the one they had previously crawled and indexed when the site was still "Under Construction" and that the other 50 URL's that the Webmaster tools said it found but had not indexed may be indexed once a new crawl is performed. If both of those are true, then it is confusing to get an error message when there is nothing really wrong with the sitemap submitted. The error message makes it look like one URL was found to be valid and would be indexed in the next crawl and the other 50 URL's it found were somehow in error and would not be found or indexed in a crawl due to that implied error in the Webmaster tools sitemap message. So I guess you are also saying that as far as you can see my sitemap is OK, one URL has already been indexed, and the other 50 are all valid candidates for indexing at the next crawl instance? Is that correct? Also, thanks much for your help.
Dnyhagen
8/7/09
Dnyhagen
@emetic. . . . If Google showed one URL indexed it was that UNDER CONSTRUCTION page and no others.

What Google is saying currently is that all it knows about your site as of this morning is the following from its cache:

http://74.125.95.132/search?sourceid=navclient-ff&ie=UTF-8&q=cache%3Ahttp%3A%2F%2Fwww.engineeringfirms.com%2F

The message at the top of that cache says  "This is Google's cache of http://engineeringfirms.com/. It is a snapshot of the page as it appeared on 15 May 2009 00:03:27 GMT."

Note that Google hasn't updated its cache of your site for almost three months, but that now that you've 'gone live' with it, it'll be crawling the site in its normal crawl cycle.

It can 'see' your sitemap.xml in any case. What error is it currently reporting for your sitemap?


8/7/09
Original Poster
emetic
Here's the error message that I got:
We encountered an error while trying to access your Sitemap. Please ensure your Sitemap follows our guidelines and can be accessed at the location you provided and then resubmit.
 
There was the big red X next to it. So I assume there is some very real problem with my sitemap structure, contents, or ability to access it. I can't tell if the sitemap I submitted is in error or whether it was successfully read by the webmaster tools routine (since it reported it found 51 URL's total) and all those many URL's it found will be crawled and possibly indexed too at the next googlebot crawl for my site. At the least the error message is confusing and vague because when it says "and can be accessed at the location you provided" either it did access it at my site or it didn't access it. If it could not access it it should simply say so so we won't be guessing. If it was able to access it and the was some other error it found it should say something like "we were able to access you sitemap but found it contained errors that will need to be corrected then resubmit the sitemap". It would be even better if when it says like it did "We encountered an error..." it told you what the error was so that you could troubleshoot it, at least for the most common errors. Just saying "Please ensure your Sitemap follows our guidelines" is much too vague. There is a page of error codes that I found but the error message I got is not one listed. Again, at this point I take the error message I got to mean your sitemap is invalid or we could not access it and your Googlebot next crawl could likely fail too.
andekian dot com
9/9/09
andekian dot com
I agree and am dealing with the same issue. In my case I have submitted a sitemap index with a list of all the different sitemaps for my site. Each site map I have contains the same urls with a different lang param. Oddly I get the same error "We encountered an error while trying to access your Sitemap. Please ensure your Sitemap follows our guidelines and can be accessed at the location you provided and then resubmit." with the big red X. Yet Google still found all the sitemaps based on the error sitemap index file. I don't get it because all the sitemaps for each locale have a status of OK with a green check mark. Curious if you ever found an answer to your question regarding this issue or if you found someone to contact about it.
 
This question is locked and replying has been disabled. Still have questions? Ask the Help Community.

Badges

Some community members might have badges that indicate their identity or level of participation in a community.

 
Expert - Google Employee — Googler guides and community managers
 
Expert - Community Specialist — Google partners who share their expertise
 
Expert - Gold — Trusted members who are knowledgeable and active contributors
 
Expert - Platinum — Seasoned members who contribute beyond providing help through mentoring, creating content, and more
 
Expert - Alumni — Past members who are no longer active, but were previously recognized for their helpfulness
 
Expert - Silver — New members who are developing their product knowledge
Community content may not be verified or up-to-date. Learn more.

Levels

Member levels indicate a user's level of participation in a forum. The greater the participation, the higher the level. Everyone starts at level 1 and can rise to level 10. These activities can increase your level in a forum:

  • Post an answer.
  • Having your answer selected as the best answer.
  • Having your post rated as helpful.
  • Vote up a post.
  • Correctly mark a topic or post as abuse.

Having a post marked and removed as abuse will slow a user's advance in levels.

View profile in forum?

To view this member's profile, you need to leave the current Help page.

Report abuse in forum?

This comment originated in the Google Product Forum. To report abuse, you need to leave the current Help page.

Reply in forum?

This comment originated in the Google Product Forum. To reply, you need to leave the current Help page.