Log Reprocessing


Certain circumstances may warrant re-processing of log data, such as a DNS server being down when the processing was, incorrectly applied filters, and so on. The following document describes the proper procedure necessary to back out and reprocess webserver log data.

Please note that reprocessing logs requires the use of Urchin utilities that are only available from a command line shell environment. It is not possible to do the complete procedure exclusively from the Urchin web-based administrative GUI.

Reprocessing a Single Day:

  • In the Urchin admin GUI, edit the Profile and turn off Log Tracking under the Storage/DB tab. Be sure to click Update to save your change.
  • Under the Log Sources tab, ensure that the proper log file (s) to be re-processed are specified. The log data should only contain hits for the date(s) that you are zeroing out the statistics for.
  • Invoke a command shell on the Urchin system
  • Run the udb-sanitizer utility in the 'util' directory/folder of the Urchin distribution with the command
      udb-sanitizer -p profile-name -d YYYYMM
    where YYYYMM is the year and month containing the day you wish to reprocess
  • Select option 5, Zero out one or more days. The utility will prompt you for the correct day and will zero out the statistics for that particular day. If you have a range of contiguous days you'd like to zero you you can specify that range by using the numbers of the start and end days separated by a hyphen (e.g. 5-10 to zero out days 5 through 10 of the month). If necessary, re- invoke the utility to zero out statistics for additional days in that month if you cannot use a range.
  • Click the Run Now button under the Run/Schedule tab for the Profile to reprocess the log data
  • Reset the Log Source by changing the Log File Path back to its original setting
  • Under the Storage/DB tab in the profile edit area, turn Log Tracking back on
  • Reprocessing an Entire Month:

    The procedure for reprocessing an entire month's worth of data is identical to the single day procedure above, except when invoking the udb-sanitizer utility select Option 2, Delete this month entirely instead of Option 5.

    Additional information:

    The udb-sanitizer utility provides additional functionality for managing Urchin databases. Please see the udb-sanitizer article in the Advanced Topics->Utilities section for further information about its capabilities and usage.