/programmable-search/community?hl=en
This content is likely not relevant anymore. Try searching or browse recent questions.
Indexing problems with Google programmable search engine
0
We use Google programmable search and have some problems with it.

The programmable search searches a newspaper archive hosted by us, where each article is a separate HTML page.

We have set up sitemaps for all the years and also have a sitemap index file.
In Search-Console the sitemaps incl- sitemap index are all shown as "Successful" at the status.
The number of URLs found is also correct.

However, the index coverage shows that only 22,983 pages were indexed and 101,325 were excluded.
How does this happen?

The low indexing is also reflected in the hits returned by the search.
Many articles that are available on the server, which are also accessible via the URL in the browser, are not found.

It is difficult to give an exact number, but for generalizing search terms it can be stated that far too few hits are displayed.

What do we have to do so that everything is really searched completely?

The crawler seems to arbitrarily exclude URLs during indexing, without a reason that we can understand.
The URLs are all similar in structure.

We want the entire archive to be searched and not rely on randomness.

We are also happy to accept references to external experts.
Details
Pinned
Locked
Latest Update Latest Updates (0)
Relevant Answer Relevant Answers (0)
All Replies (1)
Relevant Answer
Hi Raif,

This forum is for queries related to Programmable Search Engine. You query is related to indexing of your contents and Index coverage reported on Google Search Console.

Please post your query in correct forum so that your query gets the required attention:

In the meantime, please review the document below for guidance:

Thanks
marked this as an answer
This question is locked and replying has been disabled.
Discard post? You will lose what you have written so far.
Write a reply
10 characters required
Failed to attach file, click here to try again.
Discard post?
You will lose what you have written so far.
Personal information found

We found the following personal information in your message:

This information will be visible to anyone who visits or subscribes to notifications for this post. Are you sure you want to continue?

A problem occurred. Please try again.
Create Reply
Edit Reply
This will remove the reply from the Answers section.
Notifications are off
Your notifications are currently off and you won't receive subscription updates. To turn them on, go to Notifications preferences on your Profile page.
Report abuse
Google takes abuse of its services very seriously. We're committed to dealing with such abuse according to the laws in your country of residence. When you submit a report, we'll investigate it and take the appropriate action. We'll get back to you only if we require additional details or have more information to share.

Go to the Legal Help page to request content changes for legal reasons.

Reported post for abuse
Unable to send report.
Report post
What type of post are you reporting?
Google takes abuse of its services very seriously. We're committed to dealing with such abuse according to the laws in your country of residence. When you submit a report, we'll investigate it and take the appropriate action. We'll get back to you only if we require additional details or have more information to share.

Go to the Legal Help page to request content changes for legal reasons.

Reported post for abuse
Unable to send report.
This reply is no longer available.
/programmable-search/threads
//accounts.google.com/ServiceLogin
You'll receive email notifications for new posts at
Unable to delete question.
Unable to update vote.
Unable to update subscription.
You have been unsubscribed
Deleted
Unable to delete reply.
Removed from Answers
Removed from Updates
Marked as Recommended Answer
Marked as Update
Removed recommendation
Undo
Unable to update reply.
Unable to update vote.
Thank you. Your response was recorded.
Unable to undo vote.
Thank you. This reply will now display in the answers section.
Link copied
Locked
Unlocked
Unable to lock
Unable to unlock
Pinned
Unpinned
Unable to pin
Unable to unpin
Marked
Unmarked
Unable to mark
Reported as off topic
Known Issue
Fixed
Marked Fixed
Unmarked Fixed
Unable to mark fixed
Unable to unmark fixed
/profile/0
false
Search
Clear search
Close search
Google apps
Main menu
Search Help Center
true
0
false