Overview
Urchin 5 provides utilities and functionality to allow all administrative operations to be performed via unattended scripts. Only the report viewing requires the web-based interface to be operational. All configuration and log processing activities can be scripted using the following utilities and techniques. For first time users, it is helpful to run the web-based administrative interface first, in order to get familiar with the terminology and capabilities of the Urchin administrative system.
Urchin 5 includes several utilities for modifying the Urchin configuration database without using the web-based administrative interface. Located in the util directory of the distribution, these utilities are:
uconf-driver uconf-export uconf-import uconf-schedule
Each of these utilities must be run from a command shell or a script, as there is no ability to execute them from the web-based Urchin administrative interface. Complete documentation for each of these utilities is available from the Urchin Help Center at http://www.google.com/support/urchin45. Here is a summary of the functional use of these utilities:
- The uconf-export utility exports the entire configuration into a file, or to standard output (stdout) if no file is specified. The format of the exported data is an XML-type format defined in the documentation for uconf-export. Each record in the exported data corresponds to a configuration record in the configuration database.
- The uconf-import utility imports the same XML type formatted data used by uconf-export into the configuration database. This tool provides functionality for importing or editing of single records, or replacing the entire Urchin configuration with the contents of a text-based configuration file.
- The uconf-driver utility performs specific actions to individual records. All parameters can be passed on one line as arguments to the script, or a file with multiple commands (one per line) can be used.
- The uconf-schedule utility updates task scheduling directives on a global bases for all configured profiles; for example to set all Profiles to run at 1:00am daily. It also has additional capabilities to run Profiles immediately with or without permanently changing the scheduled time - see the documentation for uconf-schedule for additional details on these features.
Note that you can use the uconf-export and uconf-import utilities to easily make a backup of or restore your Urchin configuration. This provides a very quick method of recovering an Urchin installation after a disk failure or other system problem. An example of this functionality:
- Save configuration for safekeeping:
uconf-export > /path/to/saved-configurations/urchin-cfg.save
- Restore Urchin configuration from a known good backup:
uconf-import -r -f /path/to/saved-configurations/urchin-cfg.save
Intended Usage
The uconf-export and uconf-import utilities are intended to provide a simple method for importing and exporting data from the Urchin configuration database using regular text files in an XML-type format. These utilities also allow you to specify the names of profiles, log sources, filters, users, etc. in configuration directives which specify access lists rather than the more cryptic record number lists that are used by the uconf-driver utility. The uconf-import utility can be used to add new records or modify existing records, but it cannot remove old records (unless the database is completely reset with the "-r" option).
The uconf-driver utility is very powerful and can be used for very specific scripting operations that may change only a few parameters in a database record, as well as performing complete record additions, modifications and deletions from the database. It can also be used for querying the configuration database for several parameters. This utility is more ideally suited for use in an environment where scripting all administrative functions of Urchin is desired, such as in automated provisioning systems or very large hosting environments where use of the Urchin administrative GUI is impractical.
Note that the uconf-driver is a lower-level utility that does not automatically maintain associations between the various database tables when working with directives that maintain cross-reference lists. When using uconf-driver to script configuration operations, please be aware that many of the tables contain directives that refer to other records, or lists of records. These directives are: ct_ulist, ct_glist, ct_llist, ct_flist, and ct_rlist, which refer to the user, group, logfile, filter, and profile tables respectively. These lists are represented as exclamation point delimited list of recnums, as demonstrated by this list of filter records:
ct_flist="!13!36!56!"
where each entry represents the recnum value of a record and is surrounded with exclamation points. For the uconf-import and uconf-export utilities, this directive would be specified as:
ct_flist="filter1,filter2,filter3"
Important! Regardless of which utility you use to manipulate the configuration, you must be careful to keep cross-references intact. For example, a Filter record has a ct_rlist which details all of the profiles that the filter applies to; and a Profile record has a ct_flist which details all of the filters that apply to this profile. Note that the uconf-import and uconf-export utilities translate and verify the lists specified in the directive for you; uconf-driver does not.
Special Usage Notes
- The uconf-driver uses exclamation point delimited lists of record numbers for directives that maintain associations with other tables (e.g. ct_flist), whereas the uconf-export and uconf-import utilities use comma-delimited lists of names in these directives. Be sure to use the appropriate list specification syntax for the utility you are using.
- If a Profile is added but no corresponding Task is added, scheduling of the Profile cannot be managed within the Urchin admin GUI interface. In addition, the Profile cannot be scheduled to run with the Urchin Task Scheduler.
When adding or editing the "ct_password" directive for use with either a User or Remote Log Source password, uconf_driver and uconf-import will automatically encrypt the password before writing it to the Urchin configuration database to ensure that passwords are not stored in clear text. For portability reasons, the encryption is in a proprietary format that is not compatible with other password encryption formats such as "crypt" on UNIX-type systems.
Examples of pseudo-code scripts to perform tasks
This section gives examples of using uconf-driver and uconf-import in scripting pseudo-code, which could be easily translated into a UNIX-type shell script, a Perl script or a Visual Basic script depending on the needs of the application. Additional examples are given in the documentation for each specific utility.
Apply a German language setting to all users
-
$user_count = `uconf-driver action=nrecords table=user`;
for ($i = 1 ; $i <= $user_count ; $i ++) {
- uconf-driver action=set_parameter table=user entry=$i
ct_language=ge;
Add a new profile/task/logsource/user
Note that in this example, uconf-import takes care of building the proper mappings so that the new profile, task, log source and user will all be properly cross referenced, e.g. the log source is associated with the profile and vice-versa, the user will be in the profile's list of authorized users and vice-versa, etc. It should also be noted that the Task is associated to the Profile because they share the same "Name=" tag.
Step 1: Some automated provisioning application creates the following text file (/path/to/text-file):
<Profile Name="www.newdomain.com">
ct_name=www.urchin.com
ct_affiliation=(NONE)
ct_website=http://www.newdomain.com
ct_reportdomains=www.newdomain.com,newdomain.com
cs_llist=newdomain.com-access-log
ct_defaultpage=index.html
cs_vmethod=0
cs_ulist=newuser
</Profile>
<Task Name="www.newdomain.com">
ct_name=www.newdomain.com
ct_affiliation=(NONE)
cr_frequency=0
cr_runnow=0
cr_enabled=off
</Task>
<Logfile Name="newdomain.com-access-log">
ct_name=newdomain.com-access-log
ct_affiliation=(NONE)
ct_loglocation=/path/to/logs/newdomain.com-access.log
cs_logformat=auto
cr_type=local
cs_rlist=www.newdomain.com
</Logfile>
<User Name="newuser">
ct_affiliation=(NONE)
ct_fullname="New User"
ct_name=newuser
ct_password=change$me
cs_rlist=www.newdomain.com
</User>
Step 2: Call the uconf-import utility to import the new profile and user into Urchin:
-
uconf-import -f /path/to/text-file