Global Filtering of Hits from Monitoring Software

Global Filtering of Hits from Monitoring Software

Overview

Most Hosting environments provide some sort of monitoring of customer webservers in order to maintain Service Level Agreements (SLAs). As a side effect, however, the hits from this monitoring can really skew the Urchin reporting for the monitored web sites - artificially inflating session, pageview, hit and byte counts.

Recommendation

In Hosting environments that employ such monitoring, it is highly recommended that a standard/global Urchin filter is applied to each customer's configured Urchin profiles to strip out the hits generated by montoring software. This is easily done in an environment where a centralized Urchin installation (managed by the hosting company) provides reporting for each customer's website(s). In dedicated/colocation environments where the customers themselves maintain an instance of Urchin on their server(s), the Hosting company should provide a sample filter that is appropriate for the monitoring being used.

To aid in the implementation of Urchin filtering, the Host and the customer should work together to create a specific page on the customer website that only the monitoring software utilizes, e.g. something like:

     http://www.customerdomain.com/healthpage.html

Examples

Example 1: Filter out the IP address for the monitoring system

      Filter Type: Exclude
      Filter Field: IP
      Filter Spec: 172\.16\.1\.1

• This will strip any hits with the IP address 172.16.1.1 out of the webserver log as Urchin is processing it.

Example 2: Filter out specific page that the monitoring system hits

      Filter Type: Exclude
      Filter Field: REQUEST
      Filter Spec: ^/healthpage.html

• This will strip any hits with a request for /healthpage.html out of the webserver log as Urchin is processing it.

Considerations

1. It may be desirable to create additional, non-filtered Profiles for the customers so they can see the actual traffic load (including the filtering) on the webserver(s).
2. The Hosting company may want to provide a Profile that provides reporting exclusively for the monitoring hits - e.g. it filters in only hits from the monitoring software. This profile could be used to show that the proper monitoring is being done and that SLAs are being met.