/webmasters/community?hl=en
/webmasters/community?hl=en
4/11/14
Original Poster
Roxana S.

Robots.txt Paradox in Webmaster Tools

I've read the FAQs and searched the help center. 
My URL is:

Hello!

I have a bit of a paradox in WMT.
Real situation:
There's a robots file uploaded set to exclude all the pages of a website.
When I tried to fetch this robots file as Google I got a message saying "Denied by robots.txt". (image attached)

Hypothetical situation:
I understand getting this message for any other page but if I make a change to my robots and want to fetch as Google to let Google know of my changes I'll get the same denied message.

So my only option would be to wait for Google to revisit my robots file.

Since I have it on disallow all, it's most likely a new website with no links and I'm still working on it, Google won't come back too fast to check my robots file.

So what I'm saying is that maybe someone at WMT can consider that the robots.txt file should be fetched no matter what the previous robots file said in order to be able to ask Google to faster recrawl a newly constructed website that was just finished and is ready to go live.


Community content may not be verified or up-to-date. Learn more.
Recommended Answer
Was this answer helpful?
How can we improve it?
This question is locked and replying has been disabled. Still have questions? Ask the Help Community.

Badges

Some community members might have badges that indicate their identity or level of participation in a community.

 
Google Employee — Google product team members and community managers
 
Community Specialist — Google partners who help ensure the quality of community content
 
Platinum Product Expert — Community members with advanced product knowledge who help other Google users and Product Experts
 
Gold Product Expert — Community members with in-depth product knowledge who help other Google users by answering questions
 
Silver Product Expert — Community members with intermediate product knowledge who help other Google users by answering questions
 
Product Expert Alumni — Former Product Experts who are no longer members of the program
Community content may not be verified or up-to-date. Learn more.

Levels

Member levels indicate a user's level of participation in a forum. The greater the participation, the higher the level. Everyone starts at level 1 and can rise to level 10. These activities can increase your level in a forum:

  • Post an answer.
  • Having your answer selected as the best answer.
  • Having your post rated as helpful.
  • Vote up a post.
  • Correctly mark a topic or post as abuse.

Having a post marked and removed as abuse will slow a user's advance in levels.

View profile in forum?

To view this member's profile, you need to leave the current Help page.

Report abuse in forum?

This comment originated in the Google Product Forum. To report abuse, you need to leave the current Help page.

Reply in forum?

This comment originated in the Google Product Forum. To reply, you need to leave the current Help page.