/webmasters/community?hl=en
/webmasters/community?hl=en
3/7/09
Original Poster
Google user

How long till my site gets Indexed // How long for Googel to Index a site ::: Auto-Response :::

This is an Auto-Response for the (very) common question regarding Indexation Times.
This is an attempt to compile information for questions/issues such as;

Q: How long does it take to get Indexed.
Q: How long does Google take to Index a site.
Q: How fast does Google Index a site.
Q: Still not Indexed.
Q: How long does it take to get Crawled.
Q: How long does Google take to Crawl a site.
Q: How fast does Google Crawl a site.
Q: Still not Crawled
.

=============   How long to get indexed   =================

There is No set time in for Google to initially index your site - the time taken can vary.
The time it does take may vary based upon factors such as;

   * Popularity of the site (Whether it has any links to it)

   * Whether the content is Crawl-able (Server Responses and Content type)

   * Site structure (how pages interlink)

It is possible for a site to be crawled/indexed within 4 days to 4 weeks.
In some cases, it may take longer.



## Things to do to check/ensure that things are "optimal" for crawling/indexing ##

   * Check Server responses
     Pages should return 200/OK responses.
     Use tools like   http://oyoy.eu/page/headers/  to check the response.

   * Have actual "content"
     This means having content that bots/spiders can "see" and understand (text!).
     If you are using purely Images, solely Flash or loading content up via JavaScript - then you are likely to encounter problems.
     Remember - bots like to "read" - so ensure that you have "text" for them to read!

   * Use "normal" links
     Normal  links are things such as    <a href="some_URL" title="optional"> Some Link Text </a>
     Using JavaScript based events (no href value, instead using on.click or using select list elements with on.click functions etc.) may result in the bots not following!

   * Submit the site
     NO - Verification is NOT the same as Submitting.
     To submit your site - use the URL Submittal tool   http://www.google.com/addurl/?continue=/addurl

   * Be discovered
     If at all possible, have a few links to the site from other sites.
     These links should be from related/topical/relevant sites.
     These links should be from quality/trusted/popular/reliable sites.
     Links do seem to help - but do NOT go overboard - it only needs 1 link that is "normal" (can be seen/followed by bots).

   * Verify your site
     Sign up for a Google Webmaster Tools account.
     This does NOT mean the same as Submitting your URL!
     This MAY NOT improve the chances of getting crawled/indexed.
     This can help in discovering/locating/understanding problems as/when/if they occur.

   * Provide a Sitemap
     This is completely Optional!
     Sitemaps do NOT guarantee getting Crawled/Indexed.
     Sitemaps MAY NOT increase your chances of getting crawled/indexed, nor improve the speed/rate.



## Common Problems/Causes for initial Crawling/Indexing ##

   * Poor Server responses...
       - The server sending Bad Responses (404/410/500 etc.),
       - The server being Slow/Timing out (taking ages to acknowledge/reply to the server request),
       -  Having Excessive Redirect Strings (Redirect from URL A to B, to C, to D etc.),
       -  Using non-standard Redirects (JavaScript etc.),
     ...may result in the initial crawl getting stuffed.
  
   * DomainNameServer responses...
     ...that fail to resolve or that have major issues can cause problems.

   * Really shoddy code/markup...
     ...such as seriously malformed <head></head> sections etc. can cause problems.

   * robots.txt ...
       - being malformed,
       - being incorrectly setup,
       - not returning a proper response code,
     ...can cause crawling issues
  
   * robots meta ...
     ...people may have unwittingly blocked bots, included multiple/additional robots tags and unknowingly be denying bots to ability index/follow pages.



## Additional ##

Please, before deciding that your site is Not Indexed ...

1) Be aware that information/data in the Google WebMaster Tools (GWT/GWMT) may not be "accurate".
    (It is known to be out of date / out of synch in many cases)

2) To check for your site in the Search Engine Results,
    a) please use the Site: operator  =  in Google search, in teh search box, type in    site:www.yourdomain.tld   or   site:yourdomain.tld)
    b) try doing a search for the <title> of a given page using quotes around it (  "your pages title"  )
    c) try doing a search for a snippet of "unique" content using quotes (  "just copy and paste a short sentence of so from one of your pages"  )

3) If requesting help/assistance/information regarding your site and getting crawled/indexed - PLEASE supply relevant information.
    Tell us when the site went live, whether you have actually Submitted the site, whether it is Verified, whether you supplied a Sitemap, whether you access to your Server Logs, whether you can see the GoogleBot crawling your site and what response code it gives etc. etc. etc.

====================================================


NOTE:

This is a "general auto-response" post.

This is not a Topic for discussion - it is a point of reference to save having to type the same answer repeatedly due to the sheer number of times this question is asked and is meant as an aid for people that don't seem search/read the various other posts regarding this topic.
Thank you for taking the time to read this Auto-Response.
Community content may not be verified or up-to-date. Learn more.
Recommended Answer
Was this answer helpful?
How can we improve it?
All Replies (52)
Tim Abracadabra
3/7/09
Tim Abracadabra
Thanks for the great input!
 
A checklist of sorts eh?
Definitely something for visitors to review.
 
What helps our posters better understand their concerns and
how to present the best relevant information, I would think should
help to resolve issues on this topic quicker with better accuracy.
 
It might also be helpful in the AI KB tool that is in progress.
 
Much appreciated,
Tim
 
3/7/09
Original Poster
Google user
?   :scratches head:  whats a   "AI KB" ?
(Is it an Artificially Intelligent Kill-Bot?)

Would be nice if people actually read this - but mostly it's hear so I don't have to repeat my self twice a day regarding indexation because people didn't read/search :D
Tim Abracadabra
3/7/09
Tim Abracadabra
Sorry Auto, I really shouldn't abbreviate such things.
 
KB=Knowledge Base consisting of rules and predicate logic with inference.
 
Like a tool (some) users may be able to use to get answers to FAQ?
 
:-)
 
Sound vaguely familiar?
 
Tim
 
 
3/7/09
Original Poster
Google user
Well, I'm still running with ideas for a the Bionics what not.... (just need time to implement things!).
But, I'm now 100% convinced that in the main, it will be wasted.
People cannot be bothered to Read the FAQ or other topics, I don't see them spending a whole minute answering a few questions to help identify a problem, nor following links to other information.

That is apparently our job.
Answer a question several times, then simply point people to those answers!
Tim Abracadabra
3/7/09
Tim Abracadabra
>>>> Would be nice if people actually read this - but mostly it's hear so I don't have to
>>>> repeat my self twice a day regarding indexation because people didn't read/search :D
 
If others read (find) this post and find it helpful well..
 
I'm all heads and tails for it :-)
 
I've bookmarked the thread for future use ;-)
 
Tim
46 MORE
Anna Chen
6/30/10
Anna Chen
Facts are nothing without their nuance.
 
This question is locked and replying has been disabled. Still have questions? Ask the Help Community.

Badges

Some community members might have badges that indicate their identity or level of participation in a community.

 
Expert - Google Employee — Googler guides and community managers
 
Expert - Community Specialist — Google partners who share their expertise
 
Expert - Gold — Trusted members who are knowledgeable and active contributors
 
Expert - Platinum — Seasoned members who contribute beyond providing help through mentoring, creating content, and more
 
Expert - Alumni — Past members who are no longer active, but were previously recognized for their helpfulness
 
Expert - Silver — New members who are developing their product knowledge
Community content may not be verified or up-to-date. Learn more.

Levels

Member levels indicate a user's level of participation in a forum. The greater the participation, the higher the level. Everyone starts at level 1 and can rise to level 10. These activities can increase your level in a forum:

  • Post an answer.
  • Having your answer selected as the best answer.
  • Having your post rated as helpful.
  • Vote up a post.
  • Correctly mark a topic or post as abuse.

Having a post marked and removed as abuse will slow a user's advance in levels.

View profile in forum?

To view this member's profile, you need to leave the current Help page.

Report abuse in forum?

This comment originated in the Google Product Forum. To report abuse, you need to leave the current Help page.

Reply in forum?

This comment originated in the Google Product Forum. To reply, you need to leave the current Help page.