Briann - (bendruomenė)

So about about those pages that have been deleted/redirected but still appear as indexed and show in search? How do I force Google to visit those pages and remove them from search?

Should I use to "Removal Tool" for deleted/redirected pages?

Google Search Central•2023-03-06

My site has gone through several changes in content for the last couple of months. Basically, I deleted low-quality and unrelated pages, some I deleted permanently (410) others I redirected to other relevant pages (301).

My question is should I use the "Removal Tool" on those pages so that I speed up the process of Google crawling and removing those URLs from search?

I'm asking because I see that most deleted pages haven't been updated in GSC.

Like some pages that have been removed 1 month ago are showed in GSC as "Page is not indexed: Crawled - currently not indexed" and the last crawl was done like 5-6 months ago.

Those pages should be either marked as 301 or 404.

At this speed it will take ages for Google to reflect every change I made to the website.

Because I'm pretty sure these pages hurt my site, I really want to make Google see that I removed them so they don't weigh them when it comes to ranking.

Any thoughts?

5 atsakymai

2 rekomenduojami atsakymai

Atsakyti į: „What would happen if I cancel the "Change of address" after 1 month?“

Google Search Central•2023-01-25

Hi,

My question was if I should cancel the 2nd request or leave it as it is.

I check the server logs every day, nothing is wrong.

Every redirect is working and done perfectly.

Here's the GSC data maybe it says something to you:

What would happen if I cancel the "Change of address" after 1 month?

Google Search Central•2023-01-25

Hi,

I triggered the "Change of address" two times. Once 7 months ago and once again 1 month ago. I did it because the traffic didn't recovered and I thought triggering another "Change of address" will improve something. Well it didn't. Ranking got actually worse. Maybe this has caused the drop or shouldn't have any impact anything at all?

Should I cancel the 2nd request? Since I triggered the crawling rate didn't improved. In the new property in GSC in Settings appears 3 movies sits:

olddomain.con (from the 2nd request)

www.olddomain.com

old.domain.com (from the 1st request)

I really need help on this. Thank you!

3 atsakymai

Atsakyti į: „Should I request another "Change of address" after 180 days?“

Google Search Central•2023-01-03

But why duplication appeared? May be you have used 302 instead of 301? Or something another is incorrectly migrated. Or may be something else ... without URls it is difficult to me to answer.

The redirect is done correctly. Google reports these URLs as duplicate on the old domain (they appear fine on the new domain, indexed and crawled) because it didn't crawl them not even once after the domain migration. Like, I did the domain migration on July and the last crawl on these particular pages on the old domain has been in May and June. Hence, Google hasn't seen the redirect and the new canonical and marked the pages as Duplicate because at the time of the crawl the "User-declared canonical" was the old domain and now Google chose the canonical URL the new domain. Obviously, the canonical URL is the new domain but because Google didn't crawl the page it didn't see it.

I keep trying the manually index these pages every week but the report does not update.

Check internal linking - have you changed it to new URLs? During migration all internal linking should be changed.

Yes, I have updated all internal links with the new domain.

About these duplication: check canonical tag implemented by you (user) and what Google took as a canonical - same address or another?

The current canonical for these pages is the new domain. Howerver in the report it states that the "User-declared canonical" is the old domain and Google choose the new domain as canonical so it marked the pages as duplicate. The problem is that Google hasn't crawled these pages after I done the migration so it didn't confirmed the redirect and the new canonical. Again, this is only happening if I inspect the URLs on the old domain. On the new domain, these pages are fine.

How can I solve this? How can I force Google to crawl these pages. No matter how many times I trigger the "manual indexing" nothing happens. The report does not update and these pages are still marked as duplicate on the old domain.

Atsakyti į: „Should I request another "Change of address" after 180 days?“

Google Search Central•2023-01-03

Hi gonz0, I'd prefer not to. I'd love the have these questions answered by you:

Do you think it is still possible to recover after 6 months?
Do you think this duplicate issue could impact the migration?
Should I trigger another "Change of address"?

Atsakyti į: „Should I request another "Change of address" after 180 days?“

Google Search Central•2023-01-03

Yes, everything redirects to the new domain 1:1 except for the sitemap. I have left the old sitemap in place on the old domain for crawling's sake.

*I have updated all internal links

*Fixed all broken links

*Updated all social media links

*Updated the most impactful backlinks

Everything looks flawless in GSC except for 10 pages on the old domain.

These 10 pages are marked as "Duplicate content" by Google because Google hasn't crawled these pages not even once after these 6 months post domain migration. The report states these 10 pages have been last crawled 7 months + ago and Google saw the "User-declared canonical URL" as the old URL at that time in the meanwhile he choose the canonical URL the new domain but it didn't updated the "user-declared canonical", hence the duplicate content.

No matter how many times I trigger the "manual crawling" in GSC for these pages the report does not update (even though I can clearly see in logs that Google entered these pages on the old domain and it has been redirected to the new domain).

I also requested the "Validation" for these 10 pages marked as Duplicate like 2 months ago but literally, nothing happened. It likes the old property in GSC is frozen, nothing updates anymore.

Do you think it is still possible to recover even after 6 months?
Do you think this duplicate issue could impact the migration?
Should I trigger another "Change of address" so that Google would crawl these 10 pages again?

Atsakyti į: „Should I request another "Change of address" after 180 days?“

Google Search Central•2023-01-03

Bump, please does anyone has an answer for this?

Should I request another "Change of address" after 180 days?

Google Search Central•2022-12-30

I did a domain migration 6 months ago and I just saw the "moving" mark is gone. The traffic isn't recovered so basically that's it? I don't know, I simply can't accept the fact that the site won't recover and the process is done and nothing will change in the near future. Something is wrong, 80% of pages haven't appeared in search not single once in these 6 months while other pages still take the first spot on some competitive keywords. It's out of my power.

Will triggering another "Change of address" help in any way?

8 atsakymai

Atsakyti į: „How to do the "Change of address" on domain migration“

Google Search Central•2022-12-15

As an offtopic, I was temporarly banned from the platform because of this thread, I don't know why? If I do anything wrong I don't intend to, so please let me know if I violate any of the policies. I was a bit surprised but it looks like the "ban" was reversed. My continues threads about my current problems with the migration could look like spam, maybe? But I'm really desperate with the situation and I try to keep asking questions until I find a solution.

Ontopic: I understand everything you say but the domain migration I did was just to move to the plural version of the domain. It's basically the same brand name. I did everything by the book and still lose the traffic and have big problems recovering it.

And it looks like the domain migration is a big deal just for Google. Other search engines have no problem with it. I don't understand where all these concerns come from. A domain migration should come with no compromises if done correctly. If it's 100% the same site there shouldn't be any logic explanations for hindering the recovery process.

Anyway, thanks a lot for your input. So it's normal to have no "moving site" reported in the GSC for the www. version of the new URL since I triggered the change of address to the non www version.

Atsakyti į: „How to do the "Change of address" on domain migration“

Google Search Central•2022-12-14

But I seen people and an actual case study that claims it's super important to trigger the Change of address on all URLs. And I don't know exactly if I should change the www version of the old domain to the www version of the new one or to the one I prefer to have indexed.

It's super annoying that Google does not have a comprehensive guide on every detail about domain migration and also that does not notify you if anything goes wrong. I say this because I noticed and also saw almost everyone saying that Google is more problematic with domain migration than Bing or any other search engine. Literally every small detail that's not mentioned in the guidelines if done wrong can hinder the recovery process. The process is in essence super straightforward, it should be enough to redirect and request the change of address, which is for other search engines, but it looks like Google needs more than that to take the move for granted.

Saw plenty of cases where the migration was done flawless but still took a long time to recover completely, and in others, never recovered. This is alarming.

And those people that say traffic is usually recovered after 1 month or so probably never did a domain migration in their life or have moved very small sites or authority ones but no in between.

How to do the "Change of address" on domain migration

Google Search Central•2022-12-14

I didn't give it much importance after I moved the domain and initially I only requested the "Change of address" at the Domain property level. After 2 months (I think) I noticed the www version hasn't been moved along (I confirmed that the tool does not move any subdomains below the specified domain, including www). Of course, I requested the "Change of address" on the www version too but to the non-www version of the new domain.

Now after 5 months (traffic is still not recovered) I notice that the www version of the new URL has no "moving site" to it, I guess this is because I should have changed the www version of the old domain to the www v version of the new domain?

Could this impact the domain migration in any way? I read a case study claiming this is actually very important. My site hasn't been recovered even to this day so I take anything into consideration.

4 atsakymai

1 rekomenduojamas atsakymas

Google no longer index paginated pages.

Google Search Central•2022-12-13

Google used to index some paginated pages a while ago but now I see all category and tags pages are no longer indexed, just the main URL. This is not really a problem for me but I was wondering if it's a signal of low quality or technical problem.

If I inspect those paginated pages usually the first 4-5 pages return "Crawled and not indexed" and everything beyond that appears as "Page is not indexed: URL is unknown to Google"

Any thoughts?

1 atsakymas

1 rekomenduojamas atsakymas

Atsakyti į: „Need help recovering after domain migration. Google does not read/crawl the old sitemap?“

Google Search Central•2022-11-30

Hi,

May I ask what are your thoughts about the thing that Google does not read/consider the old sitemap? I think if I manage to make Google crawl the sitemap things will improve considerably.

Why is that?

- Is see on the old domain that some pages have been last crawled on June (right after I did the domain migration). Some of these pages are also marked as "Crawled not indexed" or "Page is not indexed: Duplicate, Google chose different canonical" instead of "Page with redirect" since they haven't been crawled for ages. This is another signal that Google does not read the sitemap.

- The crawling rate on images is deadly slow (3-5 crawls per day) and I have more than 5000 images on the old domain that needs to be crawled and marked as redirect. Considering that Google has no sitemap, it has no source to see the old URLs of images and confirm the redirect.

- Google keep crawling the category pages on the old domain over and over again. If it would read the sitemap I'm sure it would visit inner pages and images as well.

- All old URLs are missing the "Referring page" or "Sitemap" in the "Inspection URL" report. That clearly indicates that Google visit the old URLs from completely other sources and this may considerably slow down that crawling process.

I saw few people claiming that is a super bad idea and could break the process if you redirect the sitemap too after domain migration (which I did). Is it true?

So for the first three months after migration, all old sitemaps have redirected to the new sitemaps, so basically I had no mapping for the old URLs at that time.

In September, however, I added an old sitemap on the old domain and leave it for one week after I deleted it (I was scared that it would create a conflict between the old and new URLs). After doing that I saw a considerable increase in the crawling rate on the old domain and I also noticed that the sitemap appeared as "Reffering page" for a couple of old URLs when I inspected them. This indicates that Google indeed read the sitemap at that time and crawled a part of the URLs.

I tried to do the trick again on Octomber and again on November but it didn't worked at all. Google has stopped read the sitemap after I deleted it in September.

What do you think?

Atsakyti į: „Need help recovering after domain migration. Google does not read/crawl the old sitemap?“

Google Search Central•2022-11-29

Before the domain migration, the website was ranking on plenty of keywords. Day 4-5 after the migration 95-98% of pages have dropped from the search and only 2-5% have kept their position (the exact same pages that rank now and have ranked all the time after migration).

I think it has nothing to do with algo changes. I haven't seen any movement in clicks/impressions during the updates so they didn't impact the website at all.

And even if it was like you said, the 95% of pages that are vanished from Google are clearly good enough to be at least in the top 100 to receive impressions. Some pages that are now 1st on Google have poorer quality and more aggressive competition than other pages that haven't seen the light after migration.

Also, if the website would have been affected by any update I don't think Google would crawl the new pages in a few hours.

Atsakyti į: „Need help recovering after domain migration. Google does not read/crawl the old sitemap?“

Google Search Central•2022-11-29

Is there any chance for Google to encounter errors crawling/indexing/viewing the site without notifying in GSC or show the logs in the Coverage report? Because as I said, the Coverage report for both old and new domain is as clean as possible (all pages appear as "Page with redirect" for the old domain and "Indexed" for the new domain). I don't see any error in the "Crawl stats" either, 100% of crawls on the old domain are 301 and on the new 200 or 304.

Another "good" sign is that new content on the new domain is indexed in 1-2 hours.

But I really don't get why Google prefers to only show 5-10 pages in search (and rank them on the first page) and what stops it from showing the rest. If there would be something wrong I doubt those pages would rank at all, but they do. And this behavior of Googlebot to only visit the category pages on the old domain drives me crazy.

Atsakyti į: „Need help recovering after domain migration. Google does not read/crawl the old sitemap?“

Google Search Central•2022-11-29

I'm really hopeless. I keep asking people on all platforms but no one can figure out what is going wrong. I have almost no saving lefts because the current earnigs bearly support the costs of the website and my living (the website is practically all I do for living) and I feel so down knowing that I can't recover it from something that simple and stupid. I don't even have the money to hire someone to audit the migration..

Atsakyti į: „Need help recovering after domain migration. Google does not read/crawl the old sitemap?“

Google Search Central•2022-11-29

The old site received around 1000 clicks per day, the new site receives only 70-80. I'm saying the new site has the exact traffic it had Day1 after domain migration. It feels like the traffic and ranking keywords are capped to a static value. The new site is ranking with only a couple of keywords (those keywords haven't changed their position not once, they are literally taking the first position) while the rest 500 keywords + are vanished, don't receive a single impression. How could Google rank those pages so well on the new domain and vanish the rest?

---------------

I'm almost 90% sure that it has to do with the crawling rate on the old domain and with the fact that Google didn't checked every redirect, especially on images. It's super strange that Googlebot keeps accessing the same category URLs on the old domain over and over again while the inner pages and images are rarely crawled. Could this indicate a server problem? On Bing and other search engines I fully recovered the traffic and rankings after two weeks but Google seems to be a bit slower with the process. If there were a technical error I think I would have problems recovering the traffic on other search engines as well

Atsakyti į: „Need help recovering after domain migration. Google does not read/crawl the old sitemap?“

Google Search Central•2022-11-29

Hi,

Thanks for your input.

The website has the exact traffic and number of "ranking pages" as it was right after changing domain so I don't think it was affected by any algorithm update. I can give you the domain and server logs (in private) if you are willing to take a look at it.

Need help recovering after domain migration. Google does not read/crawl the old sitemap?

Google Search Central•2022-11-29

I have opened a couple of threads about my issues with domain migration and I'm still in the same position (clueless and hopeless). Opening a new one with a more specific question.

TL;DR - Did a domain migration 3-4 months ago and traffic is still 80% less from what was before. I still have a couple of keywords ranking on 1-3st and generating the 20% traffic, but the rest of pages are basically vanished (receive 0 impression on them, like they don't even rank in the first 200 searches).

Done a lot of research on this. There's literally no error in server logs for both old and new domain. Everything looks perfect.

Howerver, I started noticing Google crawling rate on the old domain is very slow.

80% of crawls are wasted on category pages and very few crawl the inner pages and images. My guess is that this is the reason for slow recovery because Google didn't had the chance to confirm all the redirects.

It seems like Google does not read the old sitemap so it basically has no source to take all the old URLs (inner pages and images) and confirm the redirect. I think this is also the reason Google keeps crawling the category pages because it has no other source where to view the other links. The fact that there are only 5-6 requests per day on the "Crawl requests: Image" report confirms my theory that Google didn't saw the redirect on all the images (it would take years to check every image with that crawl rate)

Most old URLs also have no "Referring page" stated in the "URL Inspection" report. If Google would see the sitemap, shouldn't all URLs have it in the "Reffering page" or under the "Sitemap" section in the report?

Other than that what could possibly be the reason for the slow recovery? I have checked everything countless times and I couldn't find anything that would hinder the process.

I had 900-1000 clicks per day from Google, now I bearly have 70-80. I'm already thinking of quitting this one but I put a lot of work into it for the last 4 years and it would be super sad to waste it on such a stupid move.

I'm sure there's something to be done but I have no idea what.

I can provide server logs, GSC reports or anything else if someone is willing to take a closer look into what's going on with Google.

11 atsakymų

1 rekomenduojamas atsakymas

Atsakyti į: „No coverage stats for the old sitemap“

Google Search Central•2022-11-29

Bumping this one!

No coverage stats for the old sitemap

Google Search Central•2022-11-25

From my understanding after changing the domain, adding the old sitemap with the old URLs in GSC would come along with a coverage report that would show the number of URLs discovered as redirects.

Well, I have added the sitemap two weeks ago, it shows there are X URLs discovered, but I have no report available. It's fine on the new domain, but on the old property I have no reports for the sitemap and no choice to choose between "submitted pages" and "all known pages".

I guess it is because Google doesn't read any link inside the sitemap?

If I inspect the old URLs most of them have no "referring page" or have as referring page an old URL from the site which was linked to. Shouldn't most of them have the "referring page" the sitemap?

Am I missing something?

Could this be the cause of the slow crawling rate I'm facing on the old domain?

2 atsakymai

Atsakyti į: „Having problems recovering after changing domain“

Google Search Central•2022-11-24

I'm also considering the time factor but I'm trying to do my best to ensure nothing hinders the process.

In your case, how is the crawling on the old domain? Because in mine, 90% of crawls are on the category pages and the rest of inner pages and images. I don't understand why it has this behavior, what it sees on these category pages to keep it crawling like that (sometimes I see 20 requests in a single day on a specific category). I think this is a huge factor that slows down the recovery because Google wastes a lot of time crawling category pages that don't actually change. I'd love to see how the crawling is going on your old domain.

Atsakyti į: „Having problems recovering after changing domain“

Google Search Central•2022-11-23

Thanks. Honestly I can't ingore the old domain because I have a feeling something is wrong. There's no reason not to be recovered already since we already have all new URLs indexed.

What do you say about the fact that Google most of the times (>90%) when visits the old domain crawls the URLs of the category pages and forget about inner pages? Sometimes it crawls one category URL 20 times in a single day although nothing has changed. Why it keep visiting those URLs over and over again? What makes it trigger this behavior? I feel like something is going on.

Having problems recovering after changing domain

Google Search Central•2022-11-23

I'm hopeless and clueless. Everything looks OK, I also asked few people involved in SEO for a long time and they haven't found anything obvious, but I can't find the reason for the slow recovery. It's a small website (400-500 pages and 4x more images) and even after 2-3 months the traffic is down to hell.

All logs on the old domain show 301.

GSC also shows pages as indexed on the new domain and on the old as "page with redirect"

My guesses are:

- Probably the cache messed up the redirects on images in the first month (when I haven't checked the logs, my bad) because on the old domain I see more than 500 errors as "Crawled - currently not indexed" and all are images. Shouldn't they be also marked as "Page with redirect"?

If that's the case I really have no idea how to make Google crawl the images more frequently to confirm the redirects.

- The crawling rate is deadly slow on the old domain. Google bearly check 5-10 inner pages and 5-10 images per day.

- Strangely Google keep crawling the category and tags pages on the old domain (I see 20 requests on a single category in a single day) while inner pages are rarely crawled. On the new domain once Googlebot arrives on that category page after being redirect it doesn't do anything. It doesn't crawl any iner links on that particular category page. Could this indicate a problem?

So what do you guys say?

2 pritarimai

5 atsakymai

Atsakyti į: „Is it OK to leave the sitemap and robots.txt in place after changing domain?“

Google Search Central•2022-11-22

Thanks!

And it's OK to "revamp" the old sitemap after two months post migration?

And regarding the robots.txt, it has the same rules but I was thinking to leave the old file in place so that I can point to the old sitemap. What do you think?

Is it OK to leave the sitemap and robots.txt in place after changing domain?

Google Search Central•2022-11-21

I see a lot of debate on this. Some advice to do a blanket redirect and redirect the sitemap and robots.txt to the new address, others say to leave those unchanged on the old domain (as in not to redirect the sitemap and robots.txt) for the sake of crawling, and others say uploading a new sitemap containing the old URLs on the new domain.

What's the best approach on this?

And considering one had changed the domain address 2-3 months ago, did a blanket redirect but still lose the traffic and sees no progress so far and the crawling rate on the old domain is dangerously slow (like it will take ages for Google to crawl all the redirects), is it a good idea to add a new sitemap with the old URLs now without creating a conflict?

3 atsakymai

1 rekomenduojamas atsakymas

Atsakyti į: „Google keep accessing old redirects“

Google Search Central•2022-11-19

Even if those redirects have been changed years ago?

Google keep accessing old redirects

Google Search Central•2022-11-18

After I changed the domain name I started tracking Google to see how it crawls and index the site. Strangely, Google keep accessing very old redirects that I did on the old URL like years ago. For example, on the old domain I created a page three years ago with the URL /random-page-old-url and changed it after two weeks to /random-page-new-url. The old version of the URL literally has 0 backlinks, isn't called anywhere on the website (nor in internal links or sitemap). The new version of the URL has been canonical for ages on the old domain.

So why Google (and even other agents) keep crawling those old redirects? They shouldn't exist anymore since they aren't called anywhere on the website.

Also, when I inspect the URLs on the old domain, sometimes I see the "Reffering pages" old versions of some URLs that have been redirected for ages. Where does Google call these redirects? I really don't understand.

Could the redirects be cached in some places I'm not aware?

Or maybe the original version of the URLs (the URL that the page initially had after being published) is stored somewhere?

3 atsakymai

2 rekomenduojami atsakymai

Why Google doesn't crawl the URLs over HTTP/2?

Google Search Central•2022-11-18

Hi,

I'm using Cloudflare and I have HTTP/2 enabled, but Google keeps crawling the URLs over HTTP/1. From what I read on the internet, Google started to crawl more than a half of the URLs over HTTP/2 so I'm not sure why it still using HTTP/1 on my site. Isn't this affect the crawling performances?

1 atsakymas

1 rekomenduojamas atsakymas

Atsakyti į: „Strange server logs after domain migration.“

Google Search Central•2022-11-16

Hi, thanks for your answer, really informative.

What do you think about the fact that Google keep crawling the category pages but not the inner pages on the old domain. Given that some URLs haven't been crawled for more than a month (as per the inspection tool) shouldn't Google check the redirects again?

Strange server logs after domain migration.

Google Search Central•2022-11-16

Hi all,

I changed the domain name 2 months ago and the website is still not recovered.

I'm looking at the server logs for both old and new domain and I see some strange behavior:

- Google visits an old URL and returns a 301 redirect, however, that redirect doesn't reflect in the logs for the new domain like I don't see any log on the requested URL within that time frame when Google saw the 301. Shouldn't I see a new log on the new domain for that page returning a 200 at the exact time when Google visited the redirect on the old domain? Am I misunderstanding how logs work?

- Strangly on the old domain Google spend 90% of time crawling category pages and only 10% or so inner pages. It continues crawls category pages from hour to hour while the inner pages are forgotten. I believe this is why in GSC when I inspect the old URLs most are shown as last crawled months ago.

How can I boost the crawling rate on the old domain because I really believe this is the problem. Even after 2 months only 5-10% pages appear like they are actually in Google... the rest 90% are completely gone, 0 impressions for them.

Have I done wrong with the migration or this is how Google works?

Thanks!

3 atsakymai

1 rekomenduojamas atsakymas

Atsakyti į: „Changed domain address but Google still choose old URLs as canonical“

Google Search Central•2022-11-16

Hi Barry, yes I wouldn't be worried about that if traffic was recovered, which is not the case. Very few pages appear in search and generate traffic, while the rest (which the website used to get traffic on) receive ZERO impressions like they don't even exist

Also, is it normal that even after 2-3 months Google still chooses the old URLs as canonical for some pages? I'm thinking this is creating a conflict between the old and new URLs which causes Google not to rank any of our pages until it figure out which one is the real canonical.

(and even when Google has picked the old URL as canonical for a specific page, it still indexes both the new and old URLs of that page, is this normal?)

Another thing I have noticed is a discrepancy between the server logs and GSC reports. If I manually inspect an old URL that I see in logs that Google has recently visited, the report states it was last crawled 2 months ago (around the time when I triggered the domain change).

Any thoughts?

Should I give it more time?

Changed domain address but Google still choose old URLs as canonical

Google Search Central•2022-11-15

Hi, what is the reason that even after 2 months after I changed the domain address Google still choose the old URLs as canonical for some pages? If I insect those pages on the new property, however, everything looks fine "Page is indexed".

The site is still not recovered and I'm thinking this is the reason.

Most old URLs are still in Google if I type site:olddomain.com.

Everything is redirected, no errors in GSC for the new domain.

1 pritarimas

2 atsakymai

1 rekomenduojamas atsakymas

Atsakyti į: „Google continue crawling ads.txt on the old, deleted subdomain“

Google Search Central•2022-11-15

This is what I don't understand too.

I have deleted the subdomain long time ago but still since I have done the domain migration I see on the old property that Google keeps crawling this old subdomain ads.txt file. In the Crawl stats report there are two requests every day on this file and all return DNS error. Strangely I never uploaded an ads.txt file on that subdomain.

If I insepct the URLs there's no "Referring page" stated in the report.

Google continue crawling ads.txt on the old, deleted subdomain

Google Search Central•2022-11-15

I changed the domain adress recently but I see on the old property that Google still crawls the ads.txt file on a subdomain which I have deleted a year ago. The address return a DNS error and I don't know what to do to stop Google crawling it.

Any idea what is causing this and how to fix?

I can't recreate the same subdomain to place a .htaccess file on it and block the crawling because I changed the domain address.

4 atsakymai

Atsakyti į: „August 2022 Helpful content update feedback“

Google Search Central•2022-09-23

So nothing about https://www.stylecraze.com , https://bestreviews.com/ , https://my-best.net/ , https://www.findthisbest.com/

Those sites are ruining Google searches.

Atsakyti į: „Question regarding domain migration“

Google Search Central•2022-08-18

Hi Kusshwaha,

The whole website has been redirected 1:1, every new URLs matching the old ones.

However, I have noticed for some time that in GSC, we have quite a few pages "not indexed" because they are pages with redirects. This is because:

- For some pages, Google finds both versions of the URL with "/" and without it, and since the version with the trailing slash redirects to the version without (which is also the version in the sitemap), Google sees it with a redirect. However, the versions without the trailing slash are indexed and good, that's why I thought I shouldn't worry about it. The strange thing is how Google found those versions of the URL ending with trailing slash. We literally have no internal links pointing to them. We also tried to inspect the URLs to see the referring pages, but we couldn't find any URL ending with "/" on those pages.

- Also, since we have migrated the domain, Google sees all the redirects we did on the old domain

and shows them in Coverage as excluded due to "Page with redirect." I think this is normal since those URLs have been actually redirected. The actual and final versions of the URLs are indexed. But why does Google prefer to show those old URLs that have been previously redirected on the old domain quite a long time ago and display them as excluded pages in GSC?

Atsakyti į: „Question regarding domain migration“

Google Search Central•2022-08-15

@Lanh Đồng Of course we followed every guideline from Google.

6 months sounds insanely long for a domain migration. I hope that is not the case as we can barely "survive" that long without generating revenue.

Any insights from other webmasters that actually did a domain migration? What was your experience?

Atsakyti į: „Question regarding domain migration“

Google Search Central•2022-08-12

@MartinMoore Yes, this is exactly what I have noticed too. The traffic appears capped and Google choose to only rank few pages on top of the search while the rest remain out of 100.

I mean, out of 300 pages, 8 pages are first on Google and the rest out of 100. How Google can rank only 3% of my pages, and all those 3% be 1st while the rest can't be found anywhere on search.

And I'm sorry to hear your rankings haven't been recovered. This is really worrying.

Question regarding domain migration

Google Search Central•2022-08-12

Hello webmasters,

On 27 June, we did a domain migration, and everything went smooth for the first week. The website didn't suffer any downtime, and for the first week, the rankings were stable, and the whole traffic was redirected to the new domain.

However, after a week, we saw most of our keywords thrown out of searches, and the traffic went down from 800 to 200 users per day.

We expected that behavior, but we didn't expect it would take so long for the rankings to recover. The problem is that even after a month and a half, the traffic is still down, and most rankings haven't recovered yet.

Now we are wondering if we did something wrong in the process, and our only guess is that we messed up by wrongly redirecting the sitemap. We found on the internet that this shouldn't be a problem, but I still want to see your input on this.

So the thing is, we redirected everything from the old domain to the new domain, including the sitemap. So at the moment, there's no sitemap on the old domain with the old links. If you enter olddomain.com/sitemap.xml, it will redirect to the newdomain.com/sitemap.xml that would have all the new links. Is this an actual problem? Should I disable this particular redirect and leave the old sitemap in place? If so, in which property should I link the old sitemap? To the property of the new domain or old domain?

To be noted that there's no error in GSC related to Coverage. On the new domain property, almost all pages have been marked as "Submitted and indexed," while for the old domain, most pages are shown as "Page with redirect" in Coverage.

Other concerns we have that add more context to our issue:

- This is the strangest thing we have noticed. So before the migration, we had more than 300 ranking keywords, from which 30 keywords were in the top 3. After the migration, only a handful of keywords (like 5-6) have kept their rankings in the top 3, while the rest of the keywords have been thrown out of 100. Even at this moment, those 5-6 pages are still on top of the rankings and generate small traffic, while more than 200-300 keywords that previously had stable rankings are still out of 100, with few keywords bouncing from day to day. We would expect to see a slight and progressive increase in the number of stable keywords, but it looks like there's no big change, and still, most pages look like they have vanished from searches. How is it possible for Google to leave more than 300 pages out of 100 (pages that previously have been ranking pretty well) from a domain that sees it as good enough to rank it in the first position for other keywords? I mean, this whole time, those 5 pages are first on Google, and the rest vanished. This discrepancy is too significant.