Global Filtering of Hits from Monitoring Software

Overview

Most Hosting environments provide some sort of monitoring of customer webservers in order to maintain Service Level Agreements (SLAs). As a side effect, however, the hits from this monitoring can really skew the Urchin reporting for the monitored web sites - artificially inflating session, pageview, hit and byte counts.

Recommendation

In Hosting environments that employ such monitoring, it is highly recommended that a standard/global Urchin filter is applied to each customer's configured Urchin profiles to strip out the hits generated by montoring software. This is easily done in an environment where a centralized Urchin installation (managed by the hosting company) provides reporting for each customer's website(s). In dedicated/colocation environments where the customers themselves maintain an instance of Urchin on their server(s), the Hosting company should provide a sample filter that is appropriate for the monitoring being used.

To aid in the implementation of Urchin filtering, the Host and the customer should work together to create a specific page on the customer website that only the monitoring software utilizes, e.g. something like:

     http://www.customerdomain.com/healthpage.html

Examples

Example 1: Filter out the IP address for the monitoring system

Filter Type: Exclude
Filter Field: IP
Filter Spec: 172\.16\.1\.1

This will strip any hits with the IP address 172.16.1.1 out of the webserver log as Urchin is processing it.

Example 2: Filter out specific page that the monitoring system hits

Filter Type: Exclude
Filter Field: REQUEST
Filter Spec: ^/healthpage.html

This will strip any hits with a request for /healthpage.html out of the webserver log as Urchin is processing it.

Considerations

1. It may be desirable to create additional, non-filtered Profiles for the customers so they can see the actual traffic load (including the filtering) on the webserver(s).
2. The Hosting company may want to provide a Profile that provides reporting exclusively for the monitoring hits - e.g. it filters in only hits from the monitoring software. This profile could be used to show that the proper monitoring is being done and that SLAs are being met.