2022-11-16
Strange server logs after domain migration.
Hi all,
I'm looking at the server logs for both old and new domain and I see some strange behavior:
- Google visits an old URL and returns a 301 redirect, however, that redirect doesn't reflect in the logs for the new domain like I don't see any log on the requested URL within that time frame when Google saw the 301. Shouldn't I see a new log on the new domain for that page returning a 200 at the exact time when Google visited the redirect on the old domain? Am I misunderstanding how logs work?
- Strangly on the old domain Google spend 90% of time crawling category pages and only 10% or so inner pages. It continues crawls category pages from hour to hour while the inner pages are forgotten. I believe this is why in GSC when I inspect the old URLs most are shown as last crawled months ago.
How can I boost the crawling rate on the old domain because I really believe this is the problem. Even after 2 months only 5-10% pages appear like they are actually in Google... the rest 90% are completely gone, 0 impressions for them.
Have I done wrong with the migration or this is how Google works?
Thanks!
Išsami informacija
Informacinis pranešimas.
Šis klausimas užrakintas ir į jį atsakyti negalima.
Bendruomenės turinys gali būti nepatikrintas arba nenaujas. Sužinokite daugiau.
Categories normally have more internal links, so more references for google to recrawl
Pirminis skelbėjas Briann pažymėjo tai kaip atsakymą
Visi atsakymai
Shouldn't I see a new log on the new domain for that page returning a 200 at the exact time when Google visited the redirect on the old domain?
Not always, google won't always go to the final URL in the chain at the same time, especially if it's recently crawled the URL on the new domain and doesn't feel the need to do so again.
But the URL can just be queued and crawled later.
Perhaps worth a quick check using the URL inspection live test on an old URL, and making sure it's rendering a page from the new one. Sadly there's no reporting on the redirect, but hopefully you'll be able to see canonicals or internal links with the new domain on in the rendered html to confirm.
There's no real way to boost crawling rate above what they were doing, other than wait it out.
2 months isn't a massive time for a migration, it doesn't sound like you've done anything wrong.
Do feel free to share the old and new domains though, if you want people to see if there's anything that stands out.
2022-11-16
What do you think about the fact that Google keep crawling the category pages but not the inner pages on the old domain. Given that some URLs haven't been crawled for more than a month (as per the inspection tool) shouldn't Google check the redirects again?
2022-11-17
Categories normally have more internal links, so more references for google to recrawl
Pirminis skelbėjas Briann pažymėjo tai kaip atsakymą