/webmasters/community?hl=en
/webmasters/community?hl=en
5/28/14
Original Poster
Aaron Bradley

Has fetch behavior changed for hash bang (#!) URLs used in AngularJS / AJAX?

As of a few days ago I was able to use Google Webmaster Tools to fetch and get detailed information about a hash-bang (!#) URL like this:

And, indeed, you could actually see in the fetch results:

This is how Googlebot fetched the page.

Note that while that's obviously an example URL, the exact URL is irrelevant.  I've tried it with multiple sites I have access to in WMT, and as long as I follow the domain with a #! - it happily fetched the home page for this non-existent URL on a non-AJAX site:

(And this was true as of 22 May, as you can see in a forum post made on 25 May, which includes a screenshot of a #! fetch result.)

As of today when I enter that same URL, simply shows the path as "/".  This is, it isn't following anything after the hash (#) - meaning the only Google fetch information I'm able to retrieve about this AngularJS site is for the home page. 

Is this a bug - perhaps a unanticipated consequences to the changes announced about WMT fetch today?

Or does Webmaster Tools no longer support fetch for AJAX hash-bang URLs (the very scheme that Google came up with to enable crawling of AJAX!)?

Note that I know how hash bangs work, and I am not interested in seeing the result of the ugly (?_escaped_fragment_=) URL, but how Google handles the hash bang ... which I was previously able to do.
Community content may not be verified or up-to-date. Learn more.
All Replies (8)
cristina
5/28/14
cristina
Can you look in server access logs, what URL is accessed when you try Fetch as Googlebot?
 
5/28/14
Original Poster
Aaron Bradley
I can't see the server logs, but whether or not I see the fetch there doesn't impact whether or not WMT is capable of reporting on hash bang URLs.  In any case, I can flat out make up hash bang URLs for any site in my WMT listings, and WMT happily reports on the URL up to "#", which almost certainly means it's not trying to fetch the #! URL at all - it flips it right away in WMT.  Try it for one of your sites:  http://[domain]/#!/i-made-this-up.
Kirby Bloom
5/31/14
Kirby Bloom
Seeing the same issue... I was able to get it to produce results by placing the bot tag in the place of the hashbang... http://[domain]/?_escaped_fragment_=/i-made-this-up when using the fetch as tool, but not sure what the means for my sitemap I just submitted with all '#!'
Iris Red
6/4/14
Iris Red
I'v been trying to get a solid answer on this for over a year.   I just wish the designers of GWT could be clear about the abilities of the tool and tell us about plans for the future.  What I have found though is that the fetch as google bot tool does fetch #! urls, but will not submit them to the index.  My guess is that there is not translator in the submit tool, to look at the #! and substitute it for  _escaped_fragment_ in the indexing process.  

 It's a mistake to actually fetch the _escaped_fragment_ version in the tool and then submit that to the index, because the tool will actually index your snapshot version and users search results will land them on your snapshot.  This is very frustrating because we know the fetch tool to be so powerful for non #! urls.  It would be great for GWT developers to handle #!

Otherwise, what I found best so far is to use my site map.  The crawler does make the translation and then index the urls in my sitemap, albeit I have to wait for the crawler to come around.   
6/4/14
Original Poster
Aaron Bradley
It's a mistake to actually fetch the _escaped_fragment_ version in the tool and then submit that to the index, because the tool will actually index your snapshot version and users search results will land them on your snapshot.

+1 on this Iris Red.  It's alright to fetch the ugly URL (where the #! has been replaced with ?_escaped_fragment_=) but you don't want to take the next step of clicking submit to index for the reason you state (at least if the ugly URL redirects to a URL containing the HTML, where that redirect target 

As an aside, even in the absence of deliberately submitting the HTML snapshot to index, I'd recommend adding a rel="canonical" to all versions that point to the !# URL.  This way if the snapshot does inadvertently get indexed the search engines will probably favor the canonical URL in search results.
maeschi
6/5/14
maeschi
>>...because the tool will actually index your snapshot version and users search results will land them on your snapshot.

Not sure if that's correct as user agents of humans are not Googlebot but browsers...
 
>>This way if the snapshot does inadvertently get indexed the search engines will probably favor the canonical URL in search results.
 
True, good idea, but also not necessary as again, the user agent will send the user to the right ("real") version of the page, the snapshot will only ever be for Googlebot.
Kingsley Idehen
6/9/14
Kingsley Idehen
Google has simply opted to stick with the HTTP standard. Fragment Identifiers are local, so they don't cross the wire from Client to Server. #! is the siloist Web symbol :-)

They should never have implemented such regression and thank heavens they fixed it.  
Iris Red
6/10/14
Iris Red

Okay, It took a couple of days to verify this, but I tested fetching as googlebot  again.  I tested with both an existing _escaped_fragment_  url and a new one.  In both cases you were correct.  While google used the snapshot for indexing the content, the canonical url was displayed to the users in search results.    This is terrific, and I hope it continues to work.  Since the search engines don't guarantee display of the canonical url, I guess there is a chance that this could break, but if the site is generally "clean" I imagine the canonical url will be honored. 
Were these replies helpful?
How can we improve them?
 
This question is locked and replying has been disabled. Still have questions? Ask the Help Community.

Badges

Some community members might have badges that indicate their identity or level of participation in a community.

 
Expert - Google Employee — Googler guides and community managers
 
Expert - Community Specialist — Google partners who share their expertise
 
Expert - Gold — Trusted members who are knowledgeable and active contributors
 
Expert - Platinum — Seasoned members who contribute beyond providing help through mentoring, creating content, and more
 
Expert - Alumni — Past members who are no longer active, but were previously recognized for their helpfulness
 
Expert - Silver — New members who are developing their product knowledge
Community content may not be verified or up-to-date. Learn more.

Levels

Member levels indicate a user's level of participation in a forum. The greater the participation, the higher the level. Everyone starts at level 1 and can rise to level 10. These activities can increase your level in a forum:

  • Post an answer.
  • Having your answer selected as the best answer.
  • Having your post rated as helpful.
  • Vote up a post.
  • Correctly mark a topic or post as abuse.

Having a post marked and removed as abuse will slow a user's advance in levels.

View profile in forum?

To view this member's profile, you need to leave the current Help page.

Report abuse in forum?

This comment originated in the Google Product Forum. To report abuse, you need to leave the current Help page.

Reply in forum?

This comment originated in the Google Product Forum. To reply, you need to leave the current Help page.