Log Reprocessing

Overview

Certain circumstances may warrant re-processing of log data, such as a DNS server being down when the processing was, incorrectly applied filters, and so on. The following document describes the proper procedure necessary to back out and reprocess webserver log data.

Please note that reprocessing logs requires the use of Urchin utilities that are only available from a command line shell environment. It is not possible to do the complete procedure exclusively from the Urchin web-based administrative GUI.

Important! Please be certain that you are running Urchin 4.101 or later before running the udb-sanitizer utility. The utility included in Urchin 4.100 did not work properly and was corrected in Urchin 4.101.

Reprocessing a Single Day:

In the Urchin admin GUI, edit the Profile and turn off Log Tracking under the Advanced tab. Be sure to click Update to save your change.

Under the Log Sources tab, ensure that the proper log file (s) to be re-processed are specified. The log data should only contain hits for the date(s) that you are zeroing out the statistics for.

Invoke a command shell on the Urchin system

Run the udb-sanitizer utility in the 'util' directory/folder of the Urchin distribution with the command

udb-sanitizer -p profile-name -d YYYYMM

where YYYYMM is the year and month containing the day you wish to reprocess

Select option 5, Zero out one day. The utility will prompt you for the correct day and will zero out the statistics for that particular day. If necessary, re-invoke the utility to zero out statistics for additional days in that month.

Click the Run Now button under the Run/Schedule tab for the Profile to reprocess the log data

Reset the Log Source by changing the Log File Path back to its original setting

Under the Advanced tab in the profile edit area, turn Log Tracking back on

Reprocessing an Entire Month:

The procedure for reprocessing an entire month's worth of data is identical to the single day procedure above, except when invoking the udb-sanitizer utility select Option 2, Delete this month entirely instead of Option 5.

Additional information:

The udb-sanitizer utility provides additional functionality for managing Urchin databases. Please see the Knowledgebase article entitled Database hard close and data recovery using the udb- sanitizer for further information about its capabilities and usage.