Getting started with Programmable Search Engine

Choose sites to include in your search engine

Adding and excluding pages or sites from your search engine follow the same pattern. In both cases, you get three options, each with different behavior. 

Include sites in your search engine:

  1. From the control panel, select the search engine you want to edit.
  2. Click Setup from the menu on the left.
  3. In the Basics tab, under Sites to search, click Add. 
  4. Enter the URL you want to add and select whether you want to include any pages that match or only that specific page. See the table below for explanations if you aren't sure which one you want. 
  5. Click Save.
Option Behavior
Include all pages whose address contains this URL Any page that contains the URL is selected for inclusion or exclusion. For example, entering www.mysite.com will match www.mysite.com/about .htmlbut will not match blog.mysite.com.
Include just this specific page or URL pattern I have entered.  If you enter a specific page, such as www.mysite.com/about.html, it will only match that exact page. You can also use URL patterns, such as www.mysite.com/*about. If you aren't sure how to use URL patterns, see More on URL patterns below.

Exclude sites from your search engine:

  1. From the control panel, select the search engine you want to edit.
  2. Click Setup from the menu on the left.
  3. In the Basics tab, click Advanced under Sites to Search to expand the Sites to exclude section.
  4. Click Add under Sites to exclude.
  5. Enter the URL you want to exclude and select whether you want to include any pages that match or only that specific page. See the table below for explanations if you aren't sure which one you want. 
  6. Click Save.
Option Behavior
Exclude all pages whose address contains this URL Any page that contains the URL is selected for inclusion or exclusion. For example, entering www.mysite.com will match www.mysite.com/about.html but will not match blog.mysite.com.
Exclude just this specific page or URL pattern I have entered.  If you enter a specific page, such as www.mysite.com/about.html, it will only match that exact page. You can also use URL patterns, such as www.mysite.com/*about. If you aren't sure how to use URL patterns, see More on URL patterns below.


More on URL patterns

You can URL patterns to specific part or all of a site at once, without having to list out all of the possible pages individually.

  • Entire domain If you want to include everything on a domain in your search results, enter the domain as *.mysite.com/*. This will include any pages in the domain, for example blog.mysite.com and www.mysite.com. 
  • Entire site If you want to include an entire site, specify the site followed by a wildcard pattern. For example, www.mysite.com/* will include all content on www.mysite.com, but not content on blog.mysite.com. 
  • Parts of a site If you want to include specific parts of a site, use a URL pattern to specify which part of the site you want to include. For example, www.mysite.com/*dogs will match any URLs that start with www.mysite.com/ and contain the word "dogs," such as www.mysite.com/dogs.html and www.mysite.com/lots-of-dogs.html. On the other hand, www.mysite.com/dogs/* will only include pages that start with www.mysite.com/dogs/. It would not match www.mysite.com/dogs.html. 
  • Single pages If you want to include specific pages, you can use the page's URL. For example, you can specify www.mysite.com/mypage.html to include the contents of mypage.html in your search results.

More detailed examples are included in this table:

Pattern Description Matches Does not match
www.example.com/ Matches a single page www.example.com/
example.com/		 host.example.com
www.example.com/stamps
www.example.com/* Matches all URLs beginning with www.example.com or example.com www.example.com
www.example.com/stamps
example.com/stamps		 host.example.com/
host.example.com/stamps
www.example.com/*kites Matches all URLs that begin with www.example.com/ or example.com/ and contain the word "kites" www.example.com/kites.html
www.example.com/kites/page2.html
www.example.com/funwithkites.html		 www.example.com
www.example.com/stamps
www.example.com/product.asp*cat=Elec Matches all URLs that begin with www.example.com/product.asp and contain the term 'cat=Elec' www.example.com/product.asp?sku=20283&cat=Elec www.example.com
www.example.com/stamps
www.example.com/*kites*fly Matches all URLs that begin with www.example.com/and contain the words "kites" and "fly" ww.example.com/kites/howto/fly.html
www.example.com/fly/howto/kites.html		 www.example.com/kites/help.html
www.example.com/help/fly.html
*.example.com/* Matches all sub-domains under example.com www.example.com/stamps
host.parent.example.com/kites
example.com/kites/fly.html		 example.host.com

 

Note: URL patterns are case-sensitive.

Note: There is an account level limit of 5,000 URL patterns across all your search engines.

You may want to augment your results with general Web Search results. This includes results from anywhere on the web, but places emphasis on your personalized results. This way, even if they search for content that isn't on your site, they can find useful results. Note that results may not match the results you'd get by searching on Google Web Search.

To set an existing search engine to search the entire web: 

  1. From the control panel, select the search engine you want to edit. 
  2. Click Setup from the menu on the left and then click the Basics tab. 
  3. Go to the Search the Entire Web section and toggle this feature On or Off.
  4. Selecting On will augment your results with general Web Search results. Selecting Off will limit your results only to the sites you specified.