uconf-import: Text-based Configuration Import Utility
The Text-based Configuration Import Utility. uconf-import, provides a command line interface for importing text-based configuration records into the Urchin configuration database. The imported data is an XML-type record format, which is directly compatible with the format exported by the uconf-export utility.
The uconf-import utility is located in the util directory of the Urchin 5 distribution.
Usage of the utility is as follows:
uconf-import [-h] (prints usage message and exits) uconf-import [-v] (prints version and exits) uconf-import [-o|-r] [-f file]where:
-f specifies the path to a file that uconf-import will write to -o overwrites existing records with the same name with the data being imported -r removes all existing configuration data and replaces it with the data being importedIf neither the "-o" or "-r" arguments are specified, only new records will be written into the configuration database; no existing records will be modified or overwritten. If no command line arguments are given, uconf-import will write to the standard output.
Input format for uconf-import
The utility imports configuration records in an XML-style format. Each record begins with a <Table Name="RecordName"> line and ends with a </Table> line. The list of configuration directives associated with the record should be ordered, one per line, between the record begin/end lines. For instance, a Profile record would look something like this:
<Profile Name="help.example.com"> ct_name=help.example.com ct_website=http://help.example.com ct_reportdomains=help.example.com,www.help.example.com cs_llist="help.example.com daily log" ct_defaultpage=index.html cs_referrallevel=3 cs_timeoffset=localtime cr_logtracking=on cr_processpath=on cs_pathlevel=3 cs_vmethod=0 cs_visitortimeout=1800 cr_sessionpageview=on cs_ulist="webmaster" ct_affiliation=(NONE) cr_profiletype=Standard_Website cs_reportset=Basic_All </Profile>The valid list of Table names is:
Global Machine Filter Logfile Profile Task Affiliation Group User
Cross Linking of Records
Many records in the Urchin configuration contain directives that cross reference records in other tables. For instance, a Profile record will have a cross reference to the Logfile table for each specified Log Source; likewise, a record in the Logfile table will have a cross reference to the Profiles which utilize that Log Source. Directives of this type usually have a name like ct_Xlist, and consist of a string of comma-separated record names in other tables that the record references. When using uconf-import, the utility will verify the input data and build the proper internal cross-reference links. For instance uconf-import will properly cross-reference the following two records when it imports them. Notice the use of cs-llist in the Profile record, and cs_rlist in the Logfile record.
<Profile Name="download.urchin.com"> ct_name=download.urchin.com ct_website=http://download.urchin.com ct_reportdomains=download.urchin.com,www.download.urchin.com cs_llist="download.urchin.com daily log" cs_timeoffset=localtime cr_logtracking=on cr_processpath=on cs_pathlevel=3 cs_vmethod=0 cs_visitortimeout=1800 cr_sessionpageview=on cr_profiletype=Standard_Website cs_reportset=Basic_All </Profile> <Logfile Name="download.urchin.com daily log"> ct_name=download.urchin.com daily log ct_loglocation=/bigdisk/rawlogs/download.urchin.com/access.log cr_type=local cs_rlist="download.urchin.com" </Logfile>
Record Ordering Requirement with "-r" argument
When using uconf-import with the "-r" argument with uconf-import, it is required that the first four records in the imported configuration be as follows:
<Global Name="Access Settings"> ... </Global> <Machine Name="Process Settings"> ... </Machine> <Affiliation Name="(NONE)"> ... </Affiliation> <User Name="(admin)"> ... </User>If uconf-import does not find the first four records to be of the type and order specified above, the utility will print a warning diagnostic and exit without performing the import.
Saving and Restoring the Urchin Configuration
Using the capabilities of the uconf-export and uconf-import utilities, it is a simple process to both save and restore the Urchin configuration to a known good state. You merely need to save the output of the uconf-export utility in a file, and then reimport it at some later date with uconf-import. This allows you recover easily from corruption caused by a server crash or other catastrophic event. For example, consider the following scenario:
- As a standard operational policy, you save your Urchin configuration
regularly with the command:
uconf-export -f /path/to/backupdir/urchin_saved_config.txt
- Time goes by, bad stuff happens and the server disk crashes, requiring a complete reinstall of Urchin.
- You restore your Urchin configuration with the command:
uconf-import -r -f /path/to/backupdir/urchin_saved_config.txtNote that in most cases, you will need to have your Urchin license reset before reloading the configuration. This should be done by contacting the customer service department of your hosting provider, or by submitting a technical support request to Urchin Software if you purchased the software directly from Urchin.
- Have your Urchin license reset as above
- Copy the complete urchin_saved_config.txt file into a new "config_base.txt" file
- Edit the config_base.txt file and:
- strip out everything except the first four records:
<Global Name="Access Settings"> <Machine Name="Process Settings"> <Affiliation Name="(NONE)"> <User Name="(admin)">
- edit the first record and reset the ct_license parameter as follows:
- strip out everything except the first four records:
- Load the base config using "uconf-import -r -f config_base.txt"
- Using the web-based administration interface, reactivate the Urchin license
- Once the license is reactivated confirm that it has the proper number of profiles, log sources, etc. in the License screen of the administration interface (Configuration -> Settings -> License) or by using the inspector utility
- Now reload the rest of the config using:
uconf-import -f /path/to/backupdir/urchin_saved_config.txtNote that you do not use the "-r" option with this second command or you will overwrite the new license. The uconf-import utility will issue a few complaints about records existing and not being overwritten, but those can safely be ignored.
Using uconf-import to compact the Urchin configuration database
When records are deleted from the Urchin configuration, either with the web-based administration interface or through the use of the uconf-driver utility, the records are actually just marked as being unused and are not removed from the database itself. In environments such as hosting operations where the configuration tends to be updated frequently, it may be beneficial to completely remove these unused records. This functionality is obtained by using the "-r" argument with uconf-import utility. Simply performing the following procedure clears out all unused records in the database, thereby compacting it.
uconf-export -f /path/to/tempdir/urchinconfig.txt uconf-import -r -f /path/to/tempdir/urchinconfig.txtUpon importing, uconf-import will also do a certain amount of sanity checking to ensure the critical records are in the proper order and that the proper record cross-references are in place.
- When a ct_password directive for a Logfile or User record is imported, the supplied password must be in cleartext format or already encrypted in Urchin's proprietary encryption format. The utility will automatically encrypt cleartext passwords as it stores them in the configuration database. To allow Urchin configuration data to be moved among different operating system platforms, Urchin uses an internal encryption format that is not compatible with standard password formats such as those generated by crypt on UNIX-type systems.
- The "-r" argument should be used with caution. When using this mode, you must ensure that the input data is both comprehensive and in the correct format since the Urchin configuration is being completely overwritten by the data supplied to uconf-import. It is safest to use "-r" importing functionality only with configuration data created by the uconf-export utility.
- Script-based Configuration Management in the Integration section of the Advanced Topics area of the Urchin documentation provides a general overview of using a script-based, unattended/automated approach to managing Urchin's configuration
- Configuration Table and Directive List in the Reference area of the Urchin documentation. This document has a complete list of all the valid tables and directives that may be used with the uconf-import utility.