Overview
You may wish to process your historical logs after installing Urchin. This is easily accomplished. Simply specify a directory and a partial filename and/or wildcard (including regular expressions) in the Log Manager's Log Settings screens. NOTE: You may not use wildcards on remote HTTP and HTTPS log sources.
How to Process Historical Logs
First, add a Log Source to the system. Click on the Configuration button at left, and then the Log Manager button. On the main screen, click on the Add button at top-right. On the first screen, select Add Local Log Source, and continue. On the next screen, click Browse, which will bring up the File Browser. Locate the correct directory in the left-side window. The right-side window will display the files in the directory, and the left side will display any other directories. When you are in the correct directory, enter a partial filename and an asterisk (or other regular expression), and click the Verify button. A window will open which will show you all the matches to your pattern. Click any of the filenames to get information on the file -- location, size, modification date, and file permissions. If the pattern match is correct, click OK, and then OK again in the File Browser window.
Next, if it hasn't been already, associate this log file with a Profile by clicking the Configuration button at left, and then the Profiles button. Once the association has been completed (see Working with Profiles), click the Run/Schedule button next to the Profile in the main Profiles listing, and schedule the execution of the Profile, or click the Run Now button for immediate processing.
Urchin does not need any sort of log rotation to avoid data duplication. Urchin is equipped with a log tracking capability that ensures only new hits are processed. However, as mentioned above, logs can quickly consume large volumes of disk space, so it is a good idea to periodically compress and archive log files. Because Urchin never needs to re-read log files once they have been processed, it is perfectly acceptable to delete the log(s) after each processing run. However, many people keep logs for a specified amount of time in case they are needed for some reason, such as if a new Profile is created for that site, and historical analysis is desired.
Recommendations
Log management is not essential from the outset, but as logs grow, it becomes important. We recommend deciding on a log management plan when you initially deploy Urchin.