Processing Apache Logs with Multiple Virtual Hosts

Introduction

Organizations may have Apache web servers with multiple virtual hosts, where the logging for all the hosted websites is done to a single log file. While it is more efficient for Urchin to process individual logs for each virtual host, it is possible to configure Profiles for each virtual host that read the single combined log.

Procedure

Step 1: Make sure that your Apache webserver is configured to create the logs in the proper format for Urchin. In particular, you'll want to ensure that the following fields are being logged, in the order shown below.

    1. Remote Host [%h]
    2. Requested Host [%v]
    3. Username [%u]
    4. Date and Time (NCSA) [%t]
    5. Request (NCSA) [%r]
    6. Status [%>s]
    7. Bytes [%b]
    8. Referrer [%{Referer}i]
    9. User-Agent [%{User-Agent}i]
    10. Cookie (if you are logging cookies, e.g. for UTM) [%{Cookie}i]
Step 2: Inside the Urchin Administration interface, navigate to Configuration -> Urchin Profiles -> Log Manager, and Add a Log Source that points to your single log file. Choose "auto" as the Log Format.

Step 3: Under Configuration -> Urchin Profiles, create a new Profile for each virtual host. In the Add Profile wizard, choose "Select Existing Log Source" when you are adding the Log Source for each Profile, and select the Log Source you configured above.

Step 4: Once you have prceeded to the "Profile Wizard: Confirmation" screen, select the "More Options (Advanced)" link. Now click on the Profile Filters Tab to add a new filter. On the "Filter Wizard: Settings" screen, select "Include Pattern ONLY" as the Filter Type, "cs_host(RAW)" as the Filter Field, and put the virtual host name in the Filter Pattern field.

Step 5: Repeat steps 3-4 for each virtual host.

Considerations

While this procedure is convenient for providing reporting for multiple virtual hosts, it requires that Urchin re-process the entire logfile for each virtual host. On a large logfile, this may have significant overhead and will impact Urchin's performance. It is highly recommended that you create a separate log file for each virtual host instead.

Although this procedure is targeted specifically written for Apache, the process should be similiar for other webserver log formats that log in the same NCSA-type fields.

It is not currently possible for Urchin to automatically detect new virtual hosts and automatically configure new Profiles for them. All Profiles must be set up explicitly.

See Also