Robots.txt Paradox in Webmaster Tools1 Recommended Answer
I have a bit of a paradox in WMT.
There's a robots file uploaded set to exclude all the pages of a website.
When I tried to fetch this robots file as Google I got a message saying "Denied by robots.txt". (image attached)
I understand getting this message for any other page but if I make a change to my robots and want to fetch as Google to let Google know of my changes I'll get the same denied message.
So my only option would be to wait for Google to revisit my robots file.
Since I have it on disallow all, it's most likely a new website with no links and I'm still working on it, Google won't come back too fast to check my robots file.
So what I'm saying is that maybe someone at WMT can consider that the robots.txt file should be fetched no matter what the previous robots file said in order to be able to ask Google to faster recrawl a newly constructed website that was just finished and is ready to go live.
Some community members might have badges that indicate their identity or level of participation in a community.
Member levels indicate a user's level of participation in a forum. The greater the participation, the higher the level. Everyone starts at level 1 and can rise to level 10. These activities can increase your level in a forum:
- Post an answer.
- Having your answer selected as the best answer.
- Having your post rated as helpful.
- Vote up a post.
- Correctly mark a topic or post as abuse.
Having a post marked and removed as abuse will slow a user's advance in levels.
View profile in forum?
To view this member's profile, you need to leave the current Help page.
Report abuse in forum?
This comment originated in the Google Product Forum. To report abuse, you need to leave the current Help page.
Reply in forum?
This comment originated in the Google Product Forum. To reply, you need to leave the current Help page.