uconf-driver: Configuration Management Utility

Overview

The Configuration Management Utility. uconf-driver, provides a command line interface for administering the Urchin 5 configuration. All functionality present in the Urchin 5 administration interface is available in this utility, thus it can completely replace the use of the administration interface for managing any facet of the Urchin 5 configuration. The uconf-driver is intended for use in situations where managing the Urchin 5 configuration through automated/unattended scripts is desired. This makes it ideal for environments such as web hosting or large corporations where the amount of data that must be managed makes it impractical to manage Urchin via the standard web-based administration interface.

The uconf-driver utility is located in the util directory of the Urchin 5 distribution.

Usage

Usage of the utility is as follows:

  uconf-driver [-h] (prints usage message and exits)
  uconf-driver [-v] (prints version and exits)
  uconf-driver [-f file] [-d path] command parameters...
where:
  -f  specifies the path to a file containing uconf-driver commands
  -d  specifies an alternative path to the Urchin configuration database
  -e  specifies printing of the "entry=" field (Urchin 5.100 and later)

and command is one of the following (note that each command is all on
one line, line breaks added for readability):

  action=ntotalrecords
  action=nrecords table=tablename
  action=list table=tablename [start=startnum] [n=count]
  action=get ident
  action=add table=tablename name=recordname \
             directive=value [directive=value ...]
  action=edit ident directive=value \
              directive=value ...]
  action=delete ident
  action=get_parameter ident parameter=directive
  action=set_parameter ident directive=value

and ident is one of:

  recnum=recordnum
  table=tablename entry=tblentry
  table=tablename name=recordname

and tablename is one of:

  Global
  Machine
  Filter
  Logfile
  Profile
  Task
  Affiliation
  Group
  User
The format of a uconf-driver command is a series of name/value pairs passed as command line arguments that define the action to be taken. In particular, the action command line argument directs the behavior of uconf-driver. The nine possible actions are described in the Usage section above. Valid user-specified configuration directives begin with the prefixes "ct_", "cr_", or "cs_", but Urchin also uses directives with a "cx_" for internal purposes. These "cx_" directives should never be modified by uconf-driver utility.

Beginning with Urchin 5.100, you must now use the "-e" option to have the "entry=" field printed for uconf-driver commands that return an entire record. This was done to improve uconf-driver performance when it is used with very large Urchin configurations. Calculation of the "entry" field requires uconf-driver to search sequentially through all records in the database, which is a slow process on Urchin configurations with tens of thousands of records. When used without the "-e" argument, uconf_driver actions are very quick even with Urchin configuration databases containing millions of records.

When invoked, uconf-driver will either execute the single action specified by the command line arguments or the multiple actions contained in a file specified with the "-f" option. If no command line arguments are given and no "-f" option is specified, uconf-driver will read actions from stdin. When specifying multiple commands via stdin or the "-f" option, each line of input should represent one uconf-driver command.

A complete list of valid Urchin configuration directives is available in the Configuration Table and Directive List article.

Overview of uconf-driver Actions

Several of the uconf-driver actions require you to identify the record that is to be retrieved or modified. This identification can be accomplished in three ways. One of the argument lists below should be substituted for the ident string (see "Usage" above) wherever a command requires a record identifier.

  1. If the internal record number is known, then substitute "recnum=recordnum" for ident as the command line argument. Note that this record number is not necessarily sequential within a table.
  2. If it desired to loop through all of the records in a particular table, then specify "entry=tblentry" and "table=tablename" as the ident command line argument. The first entry defaults to "1".
  3. The exact name of the record can be specified using the "name=name argument. Be sure to use quotes if the name may contain white spaces or characters that may be interpreted as metacharacters by the shell running your command.

Description of Actions

Retrieving Data: these commands provide the ability to extract records, partial records, and record counts from the Urchin configuration.

  • "ntotalrecords" outputs the total number of records in all tables in the Urchin configuration database.
  • "nrecords table=tablename" outputs the number of records in the specified tablename.
  • "list table=tablename [start=startnum] [n=count]" retrieves multiple records from a particular table. Each record is printed on a separate line. The optional "start=startnum" argument specifies the starting number of the table entry to begin grabbing, and the "n=count" argument specifies how many entries to retrieve. For example, if a particular table has 44 records and you want to grab records 20-29, specify "start=20 n=10". NOTE: by default, this action will print 10 records. If you wish to display more than 10 records, be sure to use the "n=XXX" option. Choose a relatively large number for "n" if you want to get all records from the table.
  • "get ident" prints all information associated with the particular ident argument on a single line.
  • "get_parameter" allows you to retrieve the value for a specific directive from the record matching ident in the configuration database, and requires the "parameter=parameter" argument.
Modifying Data: the "add", "edit" and "delete" functions provide comprehensive editing ability of records in the Urchin configuration database. A directive list should also be passed along with these actions. The "set_parameter" function sets a particular directive within a record.
  • "add" requires both the "table=tablename" and "name=recordname" parameters and inserts a new record into the database with the specified set of "directive=value pairs.
  • "edit" is similar to the "add" command, except the record directives for the specified id are replaced with the new list of "directive=value pairs. This function clears all previous directives for id and adds all new directives specified on the command line.
  • "delete" deletes the record matching ident from the table.
  • "set_parameter" sets the specified "directive=value" directive in the configuration for the record matching ident.

Diagnostics Returned and Exit status

Upon completion of a command, uconf-driver will print out one of the following diagnostics based on the action parameter that was specified:

 

Action Output Description
(any) [usage msg] command line not in recognized format
(any) [no msg] command didn't perform any action, perhaps due to an out of range entry, incorrect table name, etc.
(any) -l command line parameters invalid for request type
add [recnum] record number of record that was created
delete [recnum] record number of record that was deleted
edit [recnum] record number of record that was edited
get [record] complete set of name/value directives for specified record
get_paramenter [param] value of requested directive
list [records] complete list of all name/value directives for all records in the specified table
nrecords [count] count of records in the specified table
ntotalsrecords [count] count of all records in all tables
set_parameter 0 always prints 0 on success

Please note that you must parse the runtime output from uconf-driver to determine if the command was successful. At present, the utility always exits with a status code of 0, so the exit status cannot be used to determine if the command succeeded or not.

Examples

Here is a set of example commands using the uconf-driver utility. Please note that all commands are on a single line; line breaks have been added for readability.

# Extract the total number of records from the Urchin configuration database
uconf-driver action=ntotalrecords

# Extract the number of records in the "profile" table
uconf-driver action=nrecords table="profile"

# List records 6-8 in the "logfile" table
uconf-driver action=list table="logfile" start=6 n=3

# Extract the record for user "joe" from the "user" table
uconf-driver action=get table=user name="joe"

# Add a regular non-privileged user to the configuration
uconf-driver action=add table=user name="bob" ct_fullname="Bob Jones"
             ct_password="b0bz@pw" cs_adminlevel=3

# Set the default language for the Urchin reports to English
uconf-driver action=set_parameter table=user name="bob" cs_language=en 

# Change the network port that the Urchin webserver uses
uconf-driver action=set_parameter recnum=1 ct_port=1234

Sample Bourne Shell script to add a Profile/Task/LogSource/User using uconf-driver

This is a sample working script that demonstrates how uconf-driver could be embedded in a script to automate the creation of an entire new Profile, a Log Source for it to process, a scheduled Task, and a User with rights to view the Profile.

 

#!/bin/sh
#
# Proof-of-concept Bourne shell script for adding a Profile,
# Task, Log Source and User record set to the Urchin configuration.
# The Profile will be set to run at 01:05am daily.
#
# NOTE: Line breaks have been added for readability
#
# Define the pertinent information here. Obviously, this should really be
# stuff that's parsed from the command line.
domain=mysite.com
logfile=/path/to/webserverlogs/mysite-access.log
username=userjoe
password=joepasswd
language=en
region=us

cd /path/to/urchin/util

# Add Profile
p_recnum=`./uconf-driver action=add table=profile name=$domain \
          ct_name=$domain ct_website=http://www.$domain \
          ct_reportdomains=$domain,www.$domain`

# Add Task
t_recnum=`./uconf-driver action=add table=task name=$domain \
          ct_name=$domain cr_frequency=5 cr_enabled=on cs_hour=01 \
          cs_minute=05 cs_rid=$p_recnum`

# Set proper cross reference from Profile to Task
recnum=`./uconf-driver action=set_parameter recnum=$p_recnum cs_taskid=$t_recnum`

if [ "$recnum" != "$p_recnum" ]; then
        echo "Failed to associate profile with task"
fi

# Add Log Source
l_recnum=`./uconf-driver action=add table=logfile name=$domain \
          cr_action=2 ct_name=$domain cr_type=local ct_loglocation=$logfile \
          cs_logformat=auto cs_rlist=!$p_recnum!`

# Set proper cross reference from Profile to Log Source
recnum=`./uconf-driver action=set_parameter recnum=$p_recnum
cs_llist=!$l_recnum!`
if [ "$recnum" != "$p_recnum" ]; then
        echo "Failed to associate profile with log source"
fi

# Add regular non-privileged user with access to this Profile
u_recnum=`./uconf-driver action=add table=user name=$username \
          ct_name=$username ct_password=$password ct_fullname="$domain user" \
          cs_language=$language cs_region=$region cs_adminlevel=3 \
          cs_rlist=!$p_recnum!`

# Set proper cross reference from Profile to User
recnum=`./uconf-driver action=set_parameter recnum=$p_recnum cs_ulist=!$u_recnum!`

if [ "$recnum" != "$p_recnum" ]; then
        echo "Failed to associate user with profile"
fi
exit
##
## END OF SCRIPT
##
Here is the same script, written using DOS commands.
@echo off
REM Proof-of-concept Windows batch file for adding a Profile,
REM Task, Log Source and User record set to the Urchin configuration.
REM The Profile will be set to run at 01:05am daily.
REM
REM This should work on Windows 2000, XP and 2003 Server.
REM
REM NOTE: Line breaks have been added for readability - you will need
REM to ensure that all your commands appear on one line in the script
REM
REM
REM Prompt the user for the information we need. This section could
REM be replaced using values from command line arguments instead.
REM
set/p domain=Enter domain: 
set/p logfile=Enter webserver log pathname: 
set/p username=Enter username: 
set/p password=Enter password: 
set/p language=Enter language: 
set/p region=Enter region: 

cd c:\program files\urchin\util

REM
REM Add Profile
REM
uconf-driver action=add table=profile name=%domain%  ct_name=%domain% \
             ct_website=http://www.%domain% \
             ct_reportdomains=%domain%,www.%domain% > #tmp
set/p p_recnum= < #tmp
REM
REM Add Task
REM
uconf-driver action=add table=task name=%domain%  ct_name=%domain% \
             cr_frequency=5 cr_enabled=on cs_hour=01 cs_minute=05 \
             cs_rid=%p_recnum% > #tmp
set/p t_recnum= < #tmp
REM
REM Set proper cross reference from Profile to Task
REM
uconf-driver action=set_parameter recnum=%p_recnum% cs_taskid=%t_recnum% > #tmp
set/p recnum= < #tmp
if not %recnum%==%p_recnum% echo Failed to associate profile with task
REM
REM Add Log Source
REM
uconf-driver action=add table=logfile name=%domain% cr_action=2 \
             ct_name=%domain% cr_type=local ct_loglocation=%logfile% \
             cs_logformat=auto cs_rlist=!%p_recnum%! > #tmp
set/p l_recnum= < #tmp
REM
REM Set proper cross reference from Profile to Log Source
REM
uconf-driver action=set_parameter recnum=%p_recnum% cs_llist=!%l_recnum%!` > #tmp
set/p recnum= < #tmp
if not %recnum%==%p_recnum% echo Failed to associate profile with log source
REM
REM Add regular non-privileged user with access to this Profile
REM
uconf-driver action=add table=user name=%username% ct_name=%username% \
             ct_password=%password% ct_fullname="%domain% user" \
             cs_language=%language% cs_region=%region% cs_adminlevel=3 \
             cs_rlist=!%p_recnum%! > #tmp
set/p u_recnum= < #tmp
REM
REM Set proper cross reference from Profile to User
REM
uconf-driver action=set_parameter recnum=%p_recnum% cs_ulist=!%u_recnum%! > #tmp
set/p recnum= < #tmp
if not %recnum%==%p_recnum% echo Failed to associate user with profile
del #tmp
REM
REM END OF SCRIPT
REM
Here is the same script, written using VBS. On a Windows system, you would need to use "cscript" to run this script, e.g. "cscript add_records.vbs".
'-----------------------------------------------------------------------------
' Proof-of-concept win32 cscript for adding a Profile,
' Task, Log Source and User record set to the Urchin configuration.
' The Profile will be set to run at 01:05am daily.
'
' NOTE: Line breaks have been added for readability
'
' Define the pertinent information here. Obviously, this should really be
' stuff that's parsed from the command line.
'-----------------------------------------------------------------------------

Option Explicit
'On Error Resume Next

'-----------------------------------------------------------------------------
'  Declare needed vars
'-----------------------------------------------------------------------------
Dim driverpath 
Dim tmpfilepath 
Dim domain
Dim logfile
Dim username
Dim password
Dim language
Dim region
Dim calls

driverpath  = "C:\Program Files\Urchin\util\uconf-driver.exe"
tmpfilepath = "C:\Program Files\Urchin\var\udtemp"
domain      = "mysite.com"
logfile     = "/path/to/webserverlogs/mysite-access.log"
username    = "userjoe"
password    = "joepasswd"
language    = "en"
region      = "us"

'-----------------------------------------------------------------------------
'  Declare objects
'-----------------------------------------------------------------------------
Dim shell, fso, execo, tfile
Dim recnum, p_recnum, t_recnum, l_recnum, u_recnum

Set shell = CreateObject("WScript.Shell")
Set fso   = CreateObject("Scripting.FileSystemObject")

'---------------------------------------------------------------------------
' create temporary empty file
'---------------------------------------------------------------------------
Set tfile = fso.CreateTextFile(tmpfilepath)
tfile.Close

'---------------------------------------------------------------------------
' add profile
'---------------------------------------------------------------------------
calls = """" & driverpath & """ action=add table=profile name=""" & domain & """ 
ct_name=""" & domain & """ ct_website=""http://www." & domain & """ 
ct_reportdomains=""" & domain & ",www." & domain & """ -f " & tmpfilepath
WScript.Echo calls

Set execo = Shell.Exec(calls)

While execo.StdOut.AtEndOfStream <> true
   p_recnum = p_recnum & (execo.StdOut.ReadLine())
wend

if (isNumeric(p_recnum) = False Or p_recnum <= 0) Then
   WScript.Echo "Add Profile Failed " & p_recnum
   WScript.Quit(-1)
End If
WScript.Stdout.write (p_recnum) & isNumeric(p_recnum) & vbCRLf

'---------------------------------------------------------------------------
' add Task
'---------------------------------------------------------------------------
calls = """" & driverpath & """ action=add table=task name=""" & domain & """ 
ct_name=""" & domain & """ cr_frequency=5 cr_enabled=on cs_hour=01 cs_minute=05 
cs_rid=" & p_recnum & " -f " & tmpfilepath
WScript.Echo calls

Set execo = Shell.Exec(calls)

While execo.StdOut.AtEndOfStream <> true
   t_recnum = t_recnum & (execo.StdOut.ReadLine())
wend

if (isNumeric(t_recnum) = False Or t_recnum <= 0) Then
   WScript.Echo "Add Task Failed " & t_recnum
   WScript.Quit(-1)
End If

WScript.Stdout.write (t_recnum) & isNumeric(t_recnum) & vbCRLf

'---------------------------------------------------------------------------
' Associate task back to profile
'---------------------------------------------------------------------------
calls = """" & driverpath & """ action=set_parameter recnum=" & p_recnum & " 
cs_taskid=" & t_recnum & " -f " & tmpfilePath
WScript.Echo calls

Set execo = Shell.Exec(calls)

recnum = ""
While execo.StdOut.AtEndOfStream <> true
   recnum = recnum & (execo.StdOut.ReadLine())
wend

if (isNumeric(recnum) = False Or recnum <= 0 or recnum <> p_recnum) Then
   WScript.Echo "Failed to associate profile to task" & recnum
   WScript.Quit(-1)
End If

WScript.Stdout.write (recnum) & isNumeric(recnum) & vbCRLf

'---------------------------------------------------------------------------
' Add log source
'---------------------------------------------------------------------------
calls = """" & driverpath & """ action=add table=logfile name=""" & domain & """ 
cr_action=2 ct_name=""" & domain & """ cr_type=local ct_loglocation=""" & logfile & """ 
cs_logformat=auto cs_rlist=!" & p_recnum & "! -f " & tmpfilePath  
WScript.Echo calls

Set execo = Shell.Exec(calls)

While execo.StdOut.AtEndOfStream <> true
   l_recnum = l_recnum & (execo.StdOut.ReadLine())
wend

if (isNumeric(l_recnum) = False Or l_recnum <= 0) Then
   WScript.Echo "Failed to Add Log Source" & l_recnum
   WScript.Quit(-1)
End If

WScript.Stdout.write (l_recnum) & isNumeric(l_recnum) & vbCRLf

'---------------------------------------------------------------------------
' Associate Log Source to Profile
'---------------------------------------------------------------------------
calls = """" & driverpath & """ action=set_parameter recnum=" & p_recnum & " 
cs_llist=!" & l_recnum & "! -f " & tmpfilePath
WScript.Echo calls

Set execo = Shell.Exec(calls)

recnum = ""
While execo.StdOut.AtEndOfStream <> true
   recnum = recnum & (execo.StdOut.ReadLine())
wend

if (isNumeric(recnum) = False Or recnum <= 0 or recnum <> p_recnum) Then
   WScript.Echo "Failed to associate profile width log source" & recnum
   WScript.Quit(-1)
End If

WScript.Stdout.write (recnum) & isNumeric(recnum) & vbCRLf

'---------------------------------------------------------------------------
' Add regular non-privileged user with access to this Profile
'---------------------------------------------------------------------------
calls = """" & driverpath & """ action=add table=user name=""" & username & """ 
ct_name=""" & username & """ ct_password=""" & password & """ ct_fullname=""" & domain 
& """ cs_language=""" & language & """ cs_region=""" & region & """ cs_adminlevel=3 
cs_rlist=!" & p_recnum & "! -f " & tmpfilepath
WScript.Echo calls

Set execo = Shell.Exec(calls)

While execo.StdOut.AtEndOfStream <> true
   u_recnum = u_recnum & (execo.StdOut.ReadLine())
wend

if (isNumeric(u_recnum) = False Or u_recnum <= 0) Then
   WScript.Echo "Failed to Add Log Source" & u_recnum
   WScript.Quit(-1)
End If

WScript.Stdout.write (u_recnum) & isNumeric(u_recnum) & vbCRLf

'---------------------------------------------------------------------------
' Associate user to profile
'---------------------------------------------------------------------------
calls = """" & driverpath & """ action=set_parameter recnum=" & p_recnum & " 
cs_ulist=!" & u_recnum & "! -f " & tmpfilePath
WScript.Echo calls

Set execo = Shell.Exec(calls)

recnum = ""
While execo.StdOut.AtEndOfStream <> true
   recnum = recnum & (execo.StdOut.ReadLine())
wend

if (isNumeric(recnum) = False Or recnum <= 0 or recnum <> p_recnum) Then
   WScript.Echo "Failed to associate profile width user" & recnum
   WScript.Quit(-1)
End If

WScript.Stdout.write (recnum) & isNumeric(recnum) & vbCRLf

'---------------------------------------------------------------------------
' delete temp file and exit
'---------------------------------------------------------------------------
If (fso.FileExists(tmpfilepath) = True) Then fso.DeleteFile(tmpfilepath) End If
WScript.Quit (0)

'---------------------------------------------------------------------------
' END OF SCRIPT
'---------------------------------------------------------------------------

Special Conditions of Use

Passwords

When adding or editing the "ct_password" directive for either User or Remote Log Source password, uconf_driver will automatically encrypt the password before writing it to the configuration database to ensure that passwords are not stored in clear text. For portability reasons, the encryption is in a proprietary format that is not compatible with other password encryption formats such as "crypt" on UNIX-type systems.

Managing associations between various records

Note that uconf-driver is a lower-level utility that does not map any of the directives for you. In using the utility to script some operations, please be aware that many of the tables contain directives that refer to other records, or lists of records. These directives are: ct_ulist, ct_glist, ct_llist, ct_flist, and ct_rlist, which refer to the user, group, logfile, filter, and profile tables respectively. These lists are represented as exclamation point delimited list of record numbers:

 

	ct_flist="!13!36!56!"

where each entry represents the recnum value of a record and is surrounded with exclamation points.
Important! Be sure to keep the cross- references intact. For example, a Filter record has a "ct_rlist" which details all of the profiles that the filter applies to; and a Profile record has a "ct_flist" which details all of the filters that apply to this profile.
Important! The uconf-driver uses this exclamation point delimited lists of record numbers, whereas the uconf-export and uconf-import utilities use comma-delimited lists of names. Be sure to use the appropriate list specification depending on which utility you are using.

Considerations

The uconf-driver utility is intended for advanced users who are comfortable with command line scripting, and as such it provides only minimal error/sanity checking. Exercise caution when using the utility as improper usage could result in an unusable Urchin configuration. It is strongly recommended that the Urchin configuration be backed up using the uconf-export utility before making changes with uconf-driver.

See Also:

  • Script-based Configuration Management in the Integration section of the Advanced Topics area of the Urchin documentation provides a general overview of using a script-based, unattended/automated approach to managing Urchin's configuration
  • Configuration Table and Directive List in the Reference area of the Urchin documentation. This document has a complete list of all the valid tables and parameters that can be used with the uconf-driver utility.