Log Rotation Best Practices
Overview
It is very typical in most operating environments for the system services and applications such as webservers to generate logfiles that record actions and events related to those services. In most cases, it is also standard practice for the operating system and/or applications to perform regular maintenance on the logfiles to keep the size of the logfiles in check. This prevents the logfiles from growing without bounds and eventually running out of disk space.
A common approach to managing logs is to have a regularly scheduled log rotation task that renames the existing logs with a timestamp and then restarts the service or application with a new, zero length logfile. It is also a standard practice for the log rotation task to compress the old logfiles, and to delete logfiles after a certain age or rotation cycle threshold has been reached.
In the specific case of webserver logs, the rotation is usually handled on a daily basis to ensure that the logs remain at a manageable size. In addition, a daily rotation schedule is generally a good granularity to facilitate post-processing of webserver logs with an analysis tool like Urchin. Some webservers such as Microsoft's IIS have built-in log rotation functionality, which, when enabled, will rotate logs on a daily basis by default. Other webservers such as Apache have no explicit log rotation handler, but provide tools for easily restarting the webserver (without loss of web service) to accommodate the log rotation operation (e.g. apachectl restart ).
Log Rotation Practices
The need for log rotation to avoid duplicate processing of logs has been eliminated thanks to Urchin's Log Tracking technology. This allows Urchin much greater flexibility in processing of logs, such as the ability to process "live" logs that are still being written by the webserver, or to process logs that are rotated on an manual or irregular basis.
Important Note: Unlike some previous versions of Urchin, Urchin 6 does not provide hooks for invoking a log rotation procedure or restarting a webserver after log rotation tasks have been performed, although certain post-log-processing actions are possible as described below.
While Urchin 6 operation does not require that webserver logs be rotated regularly or at all, it is recommended that a standard log rotation scheme be implemented to ensure smooth operation and to keep the Log Tracking utility from having to do a lot of unnecessary processing. It is much more efficient from both a system and application standpoint to manage several smaller logs than one very large log, as file operations tend to slow considerably as files get larger. Smaller files are also much easier to back up and restore in the event of a disk failure or other system failure.
Log rotation mechanisms needn't be overly complex -- in most cases, a simple shell script or Perl script run daily from cron on UNIX-type systems is all that is necessary. The script merely needs to rotate the existing webserver log and timestamp it (using the %Y%m%d or YYYYMMDD formats is recommended), and restart the webserver. Additional logic can be added to prune old logfiles to keep disk space usage in check.
Configuring Urchin for Use with Log Rotation
Once you have your log rotation scheme in place, it is a simple matter to configure Urchin to process your rotated log. You can either set up the Log File Path specification to use a wildcard which matches the time-stamped log filename pattern when configuring a Log Source (.e.g. access-log.* for Apache logs or ex*.log for IIS logs) or you can use Urchin's built-in timestamp pattern matching (e.g. access-log.%Y%m%d for Apache, ex%y%m%d.log for IIS). When Urchin encounters this pattern, it will substitute yesterday's date for the %Y%m%d pattern and process the log with the resulting filename (e.g. access-log.20020617). For further information on the date matching pattern, please see Wildcard & Date Substitution in Log Path.
The wildcard specification has the advantage of allowing you to place a number of unprocessed logs in a single directory and have Urchin process them the next time it runs. This is especially convenient for handling situations where the expected logfiles are not in place when Urchin runs, e.g. due to a remote webserver being down or loss of network connectivity. The disadvantage is that Urchin must open up the directory and search each log file to determine if it has already been processed, and this can induce significant overhead when many log files are resident in the directory. If you deem your log rotation scheme to be reliable, using the YYYYMMDD pattern matching scheme is a more efficient method.
You may also wish to have Urchin 6 delete or archive/compress the log once it has been processed. Different Log Destiny options can be set in the the Advanced Settings of a Log Source. For more information on these Log Destiny settings, please see Log Management.
Important! Log Destiny options should not be used with live logs that have not been rotated!
Configuring Log Rotation on UNIX-type systems
Due to the large variation operating system functionality and webserver configurations, and the high likelyhood that log rotation procedures are highly site-specific, there is no cookbook method for establishing webserver log rotation on UNIX-type systems.
Configuring Log Rotation for Windows IIS Webserver
As mentioned above, the management functions of IIS allow for automatic log rotation of webserver logs, though this functionality is not enabled by default. Please follow the steps below to configure an IIS webserver for proper log rotation. It is recommended that the logs be rotated daily, and that the log rotation be set to happen in relation to local time. By default, IIS will rotate logs at midnight GMT rather than localtime.
Under Windows 2000, you should insure that IIS webserver is configured properly to do log rotation. This is accomplished using the Computer Management function of Windows 2000. Windows NT, Windows XP and Windows 2003 Server utilize a similar procedure. To open Computer Management and establish log rotation, perform the following actions:
-
Click Start -> Settings -> Control Panel
- Double-click Administrative Tools
- Double-click Computer Management.
- Double-click on Internet Information Services
- Right-click on Default Web Site and select Properties
- In the pop-up window, select the Web Site tab
- At the bottom of the window, click on the Properties tab
- Click the Daily radio button under the New Log Time Period heading
- Click the Use local time for file naming and rollover checkbox.
This will ensure that IIS rotates the webserver logs on a daily basis just after midnight.