/webmasters/community?hl=en
/webmasters/community?hl=en
4/25/13
Original Poster
Hotconductor

Googlebot can't access your site - Server connectivity - Unreachable page

Yes I've read the FAQs and searched the help center. 
My URL is: hatfieldmedia.com

I am having a very strange issue..  Googlebot says it cannot access my site due to 'server connectivity' issues.  I am hosting with hostgator and my server has had no downtime in 67 days, and there is no ongoing maintenance.  The issue is further complicated by the fact that one minute googlebot can crawl my site, and then next it can't!

To answer a few questions that I would ask others in this situation...

1.  It is not a robots.txt issue.  I have removed/replaced my old robots.txt and .htaccess files just incase, but it made no difference.  Webmaster tools shows no errors with robots.txt fetch.  All green checks.  =)

2. DNS is green.  The site is up, no issues accessing it as a user or administrator.  Apparently ONLY Googlebot is having intermitent issues.  If it is just a Google server glitch then I really hope that I am not alone in this.  If the DNS was generating issues as well

3.  When Googlebot crawls my site it ranges from two results...

A. Success - This happens instantly and off/on.  Failure at 5am, success at 5:05am, then failure again at 5:07am.  It is ridiculous.  When success happens it is instant.

B. Unreachable page - When Googlebot fails it takes a long time (5-7 seconds) before revealing a failure.  The ONLY bit of information contained in the unreachable page result is the download time: 15000 milliseconds

So to summarize... Ever since around 1am last night, Googlebot has been getting a 33.3% failure rate at connecting with my site.  100% uptime, no issues accessing the site on my end or via support at Hostgator.

I appreciate any help you can lend.  Hostgator essentially says "we can't see any problems" and I cannot find a single thing wrong (all plugins disabled, no problems on my end or via web).  Googlebot is the only user that has an issue!
Community content may not be verified or up-to-date. Learn more.
Recommended Answers (3)
Were these answers helpful?
How can we improve them?
All Replies (237)
4/25/13
Expert - Gold
Robbo


Hi

Thanks for providing relevant details.

From what you say, I think that three main possibilities are  likely:

1.  Merely transient errors - something happened momentarily that prevented google from getting an appropriate response.
(NB.  You said ' Ever since around 1am last night, Googlebot has been getting a 33.3% failure rate at connecting with my site.'
If you mean that you have only experienced this error in the last day or so, I think it too early to identify a traceable "cause".

2.  You mentioned your "hosting" but if your domain registration is provided separately from your hosting, there could be a problem with DNS outside your hosting.

3.  If the error persists for some longer time (days/weeks) it may be that your hosting company firewall is blocking SOME google IPs and not others.   This is sometimes done accidentally or deliberately temporarily and then mistakenly forgotten.  It can even happen automatically if they have in place a system that blocks IPs simply because they making "excessive" requests/demands.



Also note that if the "errors" only started occurring in the last day or so, it may take some time for the issue to be reported in Webmaster Tools.

Suggested actions:

--- keep under review for a longer time; 

--- use Fetch as Google to check ability to access the site; 

--- take care not to misinterpret the results.   e.g. google typically requests robots.txt up to once a day;  therefore if there was a problem accessing your robots.txt some hours ago, it could still affect FaG now.  i.e. if FaG does not recognize that it is OK to crawl at this time, it may defer to another day.  That does NOT mean there is still a problem right now.



Gamechup
4/25/13
Gamechup
I have the exact same issue.

My site www.gamechup.com shows unreachable in Fetch as Google and the only problem shown there is 15000 milliseconds. This happened a few hours ago and the hosting provider says it's all fine in their end.

Please help this is affecting my indexing.
4/25/13
Original Poster
Hotconductor
Thank you for your response.  I honestly do not think there was or is any issue related to to the robots access.  The file has been wide open, and still is.  Webmaster tools has shown that aspect to be green without any issue.  It is purely under the 'server connectivity' tab.

Fetch as Googlebot is still doing the same thing...  instant access without issue (success) and failed access due to 'server connectivity' issues.

Regarding DNS, the domain is through godaddy, but they are not showing any issues at this time.

I will definitely keep an eye on it!  Hopefully this is just a glitch and will resolve itself soon!
4/25/13
Expert - Gold
Robbo


Also note that "server connectivity" includes things that do not indicate misconfiguration in the DNS or the hosting of the domain.

For example, a "timeout" due to the server being overloaded at that time. 


I'm sure we have all faced situations where a site fails to respond, but then seems to come to life again a few minutes later.


crazyaboutmagazines
4/25/13
crazyaboutmagazines
I'm having the same issue with my site. Never had this problem in all the years I've had my site. Is it site-specific.... or something else we should know about?

231 MORE
2/25/14
Expert - Gold
webado
Carlos, please open your own thread and provide all the information.
 
This question is locked and replying has been disabled. Still have questions? Ask the Help Community.

Badges

Some community members might have badges that indicate their identity or level of participation in a community.

 
Expert - Google Employee — Googler guides and community managers
 
Expert - Community Specialist — Google partners who share their expertise
 
Expert - Gold — Trusted members who are knowledgeable and active contributors
 
Expert - Platinum — Seasoned members who contribute beyond providing help through mentoring, creating content, and more
 
Expert - Alumni — Past members who are no longer active, but were previously recognized for their helpfulness
 
Expert - Silver — New members who are developing their product knowledge
Community content may not be verified or up-to-date. Learn more.

Levels

Member levels indicate a user's level of participation in a forum. The greater the participation, the higher the level. Everyone starts at level 1 and can rise to level 10. These activities can increase your level in a forum:

  • Post an answer.
  • Having your answer selected as the best answer.
  • Having your post rated as helpful.
  • Vote up a post.
  • Correctly mark a topic or post as abuse.

Having a post marked and removed as abuse will slow a user's advance in levels.

View profile in forum?

To view this member's profile, you need to leave the current Help page.

Report abuse in forum?

This comment originated in the Google Product Forum. To report abuse, you need to leave the current Help page.

Reply in forum?

This comment originated in the Google Product Forum. To reply, you need to leave the current Help page.