CPC Data Import Utilities (Urchin 6.5+)

CPC Data Import Utilities (Urchin 6.5+)

Urchin 6.5 includes new utilities for working with CPC data.

urchincpc - The Urchin CPC Data Importing tool extracts CPC data through the Google AdWords and Yahoo! Search Marketing API for further processing and reporting.

  • Note: Archived CPC data downloaded from AdWords may not exceed 250MB (Urchin 6.501+).

urchincpcd - The Urchin CPC Source Scheduler (or simply CPC Scheduler) is used to schedule CPC download tasks based on a user defined schedule (similar to the Profile Scheduler).

The CPC Scheduler (urchincpcd) runs on a local host along with the Urchin Master Scheduler (urchind). The Master Scheduler is required to be up and running in order for CPC Sources to be scheduled. The CPC Scheduler periodically checks the queue for CPC data download tasks and controls their execution.

The CPC Importing Tool (urchincpc) is automatically started if there are any scheduled tasks. The CPC Scheduler spawns worker processes internally, however, you can call them directly via the command line to start executing any tasks that are available in queue.

Downloaded CPC data is stored locally at /path/to/urchin/data/cpc/ and is automatically picked up by the Urchin engine during processing of the linked profiles.

The Urchin CPC Scheduler is managed by the urchinctl utility. To exclusively address the CPC Source Scheduler, use the "-c" option. For example,

 

urchinctl -c start/stop/restart

will start/stop/restart only the CPC Scheduler.

CPC_Data_Import

To schedule an immediate download of CPC data, go to Configuration->CPC Source Manager-> Download or Edit -> Schedule tab for a particular CPC Source and click the Download Now button. Note that CPC Scheduler must be up and running to process the request.

Usage: CPC Scheduler - urchincpcd

The CPC Scheduler is available in bin folder of your Urchin 6.5+ installation. The following options are available via the command line:

  • -d enables debug mode. More debug information is logged into output, on Windows works with -console only, e.g.

    •            urchincpcd.exe -console -d
          
  • -h prints help information and exits

  • -v prints version number and exits

For Windows, provide one of the following commands as a first argument:

  • -install to install the CPC scheduler Windows service

  • -remove to remove the CPC Scheduler Windows service

  • -start to start the CPC Scheduler Windows service

  • -stop to stop the CPC Scheduler Windows service

  • -console to start the CPC Scheduler in console mode

Usage: CPC Importing Tool - urchincpc

The following options are available via the command line:

  • -c [id] download the specified cost source, "id" is the CPC source ID

  • -d enables debug mode

  • -h prints help information and exits

  • -v prints version number and exits

Please note that if the CPC Scheduler is started without parameters or with only the -d option, it starts processing any tasks that are available in the queue and exits once there are no tasks to process.

Downloading CPC Data via proxy (6.501+)

To enable CPC Data downloading via proxy, edit the urchin.conf file, uncomment the WebServiceProxyHost, WebServiceProxyPort, WebServiceProxyUsername, and WebServiceProxyPassword entries, and provide the desired values.

 

 ######################################################
# CPC Source Manager Settings
#...
# WebServiceProxyHost, WebServiceProxyHost: Host and port of the HTTP/HTTPS
# proxy CPC Source Manager must use to access external CPC sources.
# WebServiceProxyUsername, WebServiceProxyPassword: username and password for
# proxy that enforces authorization.
######################################################
#WebServiceProxyHost: <proxy host>
#WebServiceProxyPort: <proxy port>
#WebServiceProxyUsername: <username>
#WebServiceProxyPassword: <password>

For WebServiceProxyHost, you can enter either an IP address or full domain name.