uconf-import: Text-based Configuration Import Utility

Overview

The Text-based Configuration Import Utility. uconf-import, provides a command line interface for importing text-based configuration records into the Urchin configuration database. The imported data is an XML-type record format, which is directly compatible with the format exported by the uconf-export utility.

The uconf-import utility is located in the util directory of the Urchin 5 distribution.

Usage

Usage of the utility is as follows:

  uconf-import [-h] (prints usage message and exits)
  uconf-import [-v] (prints version and exits)
  uconf-import [-o|-r] [-f file]

where:

  -f  specifies the path to a file that uconf-import will write to
  -o  overwrites existing records with the same name with the
      data being imported
  -r  removes all existing configuration data and replaces it
      with the data being imported

If neither the "-o" or "-r" arguments are specified, only new records will be written into the configuration database; no existing records will be modified or overwritten. If no command line arguments are given, uconf-import will write to the standard output.

Input format for uconf-import

The utility imports configuration records in an XML-style format. Each record begins with a <Table Name="RecordName"> line and ends with a </Table> line. The list of configuration directives associated with the record should be ordered, one per line, between the record begin/end lines. For instance, a Profile record would look something like this:

<Profile Name="help.example.com">
   ct_name=help.example.com
   ct_website=http://help.example.com
   ct_reportdomains=help.example.com,www.help.example.com
   cs_llist="help.example.com daily log"
   ct_defaultpage=index.html
   cs_referrallevel=3
   cs_timeoffset=localtime
   cr_logtracking=on
   cr_processpath=on
   cs_pathlevel=3
   cs_vmethod=0
   cs_visitortimeout=1800
   cr_sessionpageview=on
   cs_ulist="webmaster"
   ct_affiliation=(NONE)
   cr_profiletype=Standard_Website
   cs_reportset=Basic_All
</Profile>

The valid list of Table names is:

Global
Machine
Filter
Logfile
Profile
Task
Affiliation
Group
User

Cross Linking of Records

Many records in the Urchin configuration contain directives that cross reference records in other tables. For instance, a Profile record will have a cross reference to the Logfile table for each specified Log Source; likewise, a record in the Logfile table will have a cross reference to the Profiles which utilize that Log Source. Directives of this type usually have a name like ct_Xlist, and consist of a string of comma-separated record names in other tables that the record references. When using uconf-import, the utility will verify the input data and build the proper internal cross-reference links. For instance uconf-import will properly cross-reference the following two records when it imports them. Notice the use of cs-llist in the Profile record, and cs_rlist in the Logfile record.

<Profile Name="download.urchin.com">
   ct_name=download.urchin.com
   ct_website=http://download.urchin.com
   ct_reportdomains=download.urchin.com,www.download.urchin.com
   cs_llist="download.urchin.com daily log"
   cs_timeoffset=localtime
   cr_logtracking=on
   cr_processpath=on
   cs_pathlevel=3
   cs_vmethod=0
   cs_visitortimeout=1800
   cr_sessionpageview=on
   cr_profiletype=Standard_Website
   cs_reportset=Basic_All
</Profile>

<Logfile Name="download.urchin.com daily log">
   ct_name=download.urchin.com daily log
   ct_loglocation=/bigdisk/rawlogs/download.urchin.com/access.log
   cr_type=local
   cs_rlist="download.urchin.com"
</Logfile>

Record Ordering Requirement with "-r" argument

When using uconf-import with the "-r" argument with uconf-import, it is required that the first four records in the imported configuration be as follows:

<Global Name="Access Settings">
  ...
</Global>

<Machine Name="Process Settings">
  ...
</Machine>

<Affiliation Name="(NONE)">
  ...
</Affiliation>

<User Name="(admin)">
  ... 
</User>

If uconf-import does not find the first four records to be of the type and order specified above, the utility will print a warning diagnostic and exit without performing the import.

Saving and Restoring the Urchin Configuration

Using the capabilities of the uconf-export and uconf-import utilities, it is a simple process to both save and restore the Urchin configuration to a known good state. You merely need to save the output of the uconf-export utility in a file, and then reimport it at some later date with uconf-import. This allows you recover easily from corruption caused by a server crash or other catastrophic event. For example, consider the following scenario:

As a standard operational policy, you save your Urchin configuration regularly with the command:
```
  uconf-export -f /path/to/backupdir/urchin_saved_config.txt
```
Time goes by, bad stuff happens and the server disk crashes, requiring a complete reinstall of Urchin.
You restore your Urchin configuration with the command:
```
  uconf-import -r -f /path/to/backupdir/urchin_saved_config.txt
```
Note that in most cases, you will need to have your Urchin license reset before reloading the configuration. This should be done by contacting the customer service department of your hosting provider, or by submitting a technical support request to Urchin Software if you purchased the software directly from Urchin.

Important Note: If your Urchin configuration has more than 100 profiles, the recovery process requires an additional step to reactivate the Urchin license before loading the complete saved configuration. Please follow these additional steps:

Have your Urchin license reset as above
Copy the complete urchin_saved_config.txt file into a new "config_base.txt" file

Edit the config_base.txt file and:

strip out everything except the first four records:

      <Global Name="Access Settings">
      <Machine Name="Process Settings">
      <Affiliation Name="(NONE)">
      <User Name="(admin)">

edit the first record and reset the ct_license parameter as follows:
```
      ct_license=0
```

Load the base config using "uconf-import -r -f config_base.txt"
Using the web-based administration interface, reactivate the Urchin license
Once the license is reactivated confirm that it has the proper number of profiles, log sources, etc. in the License screen of the administration interface (Configuration -> Settings -> License) or by using the inspector utility
Now reload the rest of the config using:
```
     uconf-import -f /path/to/backupdir/urchin_saved_config.txt
```
Note that you do not use the "-r" option with this second command or you will overwrite the new license. The uconf-import utility will issue a few complaints about records existing and not being overwritten, but those can safely be ignored.

Using uconf-import to compact the Urchin configuration database

When records are deleted from the Urchin configuration, either with the web-based administration interface or through the use of the uconf-driver utility, the records are actually just marked as being unused and are not removed from the database itself. In environments such as hosting operations where the configuration tends to be updated frequently, it may be beneficial to completely remove these unused records. This functionality is obtained by using the "-r" argument with uconf-import utility. Simply performing the following procedure clears out all unused records in the database, thereby compacting it.

uconf-export -f /path/to/tempdir/urchinconfig.txt
uconf-import -r -f /path/to/tempdir/urchinconfig.txt

Upon importing, uconf-import will also do a certain amount of sanity checking to ensure the critical records are in the proper order and that the proper record cross-references are in place.

Considerations

When a ct_password directive for a Logfile or User record is imported, the supplied password must be in cleartext format or already encrypted in Urchin's proprietary encryption format. The utility will automatically encrypt cleartext passwords as it stores them in the configuration database. To allow Urchin configuration data to be moved among different operating system platforms, Urchin uses an internal encryption format that is not compatible with standard password formats such as those generated by crypt on UNIX-type systems.
The "-r" argument should be used with caution. When using this mode, you must ensure that the input data is both comprehensive and in the correct format since the Urchin configuration is being completely overwritten by the data supplied to uconf-import. It is safest to use "-r" importing functionality only with configuration data created by the uconf-export utility.

See Also:

Script-based Configuration Management in the Integration section of the Advanced Topics area of the Urchin documentation provides a general overview of using a script-based, unattended/automated approach to managing Urchin's configuration
Configuration Table and Directive List in the Reference area of the Urchin documentation. This document has a complete list of all the valid tables and directives that may be used with the uconf-import utility.