How the Urchin Traffic Monitor (UTM) Works
Overview
The patent-pending Urchin Traffic Monitor (UTM), originally available in Urchin 4, was specifically designed to provide the most accurate measurements of unique website visitors. For businesses looking to get a deeper understanding of their online visitor behavior, the UTM is an extremely valuable technology that combines the best of client and server side information while letting you control the data. Easy to install, this technology allows business owners to exactly identify unique visitors, click paths, and return loyalty metrics including first time visitors, returning visitors, and frequency of use. More recent versions of the UTM provide enhanced campaign tracking capability.
There are two components to the Urchin Traffic Monitor System: the UTM Sensor, which is a code snippet installed into the content of the website; and the UTM Engine, which is part of the log processing Urchin Engine. The UTM Sensor enables client-side data collection, which is then funneled back to Urchin for each pageview. The client-side information is combined with the existing server-side data by the UTM Engine to provide a more accurate and complete picture of website activity.
The UTM Sensor is a small amount of JavaScript code that accomplishes two important functions. First, the Sensor negates the effects of caching by forcing at least one hit to progress to the Google webservers for each pageview. Secondly, the UTM Sensor uniquely identifies each visitor by using client-side "1st party" cookies to keep track of first time and returning visitors. This cookie identifier is a communication tag only viewable to the web server in the same way as session ids. It is not a third party cookie, which sends information outside the system, violating many privacy policies.
The UTM System works as follows. The web server provides two basic functions: content delivery and logging. The content of the website includes the UTM Sensor which is delivered to the user's browser. The UTM Sensor sets unique identifiers and sends an additional request back to the same web server. The UTM Engine, which is part of the Urchin log processing engine, uses the data from this request to provide an accurate and more complete picture of visitor behavior.
UTM Sensor
The UTM Sensor increases the accuracy and completeness of logfile data by negating the effects of caching and proxying. For example, when the user goes back to a previously viewed page, the content of the page is delivered from the cached memory of the browser. The same model applies if the caching is provided by a service provider. The web server has no knowledge of this pageview. However, the UTM Sensor activates an additional unique hit that forces at least one small record back to the webserver. This information is logged in the logfiles, which now has knowledge of the originating pageview.
The second important function of the UTM Sensor is to uniquely identify both sessions and unique visitors. Through a patent-pending combination of browser cookies, the Sensor detects and initializes the unique visitor and session identifiers allowing exact monitoring of new and returning visitors regardless of service provider proxy behavior. Most service providers take advantage of proxying by recycling IP addresses and clustering users behind firewalls. This can cause problems with normal logfile tracking, which typically utilizes the IP address as an identifier of the user.
The UTM Sensor is able to pierce the veil of the proxy by utilizing the cookie identifiers instead of the IP addresses. For example, when a first time unique visitor accesses the website through a firewall with one IP address, the delivered pageview contains the UTM Sensor, which sets the identifier on the visitor's browser.
On the return visit by the same visitor, the unique id is passed to the web server along with each request. So even if the user is now assigned a second IP address, the UTM technology properly identifies the visitor with the original id. In addition to negating the effects of complex proxying techniques, this also tracks visitors who travel and may use their laptops from several locations and through several providers.