Custom Data Maps
Data maps describe set of tables, used as storage for data calculated from log files. Predefined tables are as follows:
- T - totals;
- TR - transactions;
- H - histogram;
- V - visitors;
- 1-n - predefined aggregate tables described in the data map file.
The default set of tables is stored in the <Urchin>/lib/reporting/profiletypes/Default/datamap.dm2 file. It uses the following format for tables:
- Each table entry uses one row and should contain the following fields: TABLE AFIELD BFIELD CFIELD DFIELD EFIELD REQUIREMENTS SPECIAL IFIELDS PPFIELDS;
- Columns should not contain spaces;
- Columns are separated by one or more spaces;
- If the value is empty it should be replaced by the '-' sign;
- TABLE - unique identifier of table, maximal value is 200;
AFIELD-EFIELD - string fields from <Urchin>/lib/reporting/logformats/fieldlist.txt;
- REQUIREMENTS describes which fields (A-E) must be present in hit for latter to get into table, kind of primary key constraint.
e.g. given that A is required in table T1, A refers to "IP-address" (cfield=4), and hit doesn't contain IP-address, as a result whole hit will be bypassed and won't get into T1. however, if another table T2 refers to "IP-address", but REQUIREMENT contains no A, hit will get to T2.
- SPECIAL - used to link records from aggregates table to records from internal tables (sessions, visitors, transaction, transaction items, paths), kind of foreign key. E.g. table #1 from default datamap file links source, medium and campaign to session (with "source" as primary key), table #2 links source, medum, term and content to session (with "source" as primary key), table #12 links request stem and page title to paths. It is not recommended to make any changes in tables with special flags or somehow use flags in custom tables, since it may lead to clobbering of foreign keys.
- S1-S11 - session link
- P1 - path link
- PARAM - special parameter
- T1-T2 - transaction link
- I0 - I1 - item link
- IPATH - initial path
- GTHRU - goal click thru
- GPATH - reverse goal path
- CTHRU - click thru
- IFIELDS - metrics. Tables store preprocessed set of metrics to optimize data extraction speed. i.e. when engine processes logs, it makes all required calculations for each unique combination of dimensions (or cfields) to avoid this phase during extraction. E.g. given cfields: IP-address, ifields: visits. during processing new table is created to store precalculated visits for each IP-address. report, that is built on top of table, extracts the data right away almost without any extra calculation effort. Valid values for IFIELDS column are described below;
- PPFIELDS - post processing fields (from range of ifields), the data for the post processed fields is determined from the session record and its associated path, transaction, and item records. Using ppfields in a custom report is not recommended.
To create custom data map entry, follow the above instructions to create a new row. To avoid breaking the data map, make sure that each new table entry uses a unique number.
Example of row in data map file:
TABLE |
AFIELD |
BFIELD |
CFIELD |
DFIELD |
EFIELD |
REQUIREMENTS |
SPECIAL |
IFIELDS |
PPFIELDS |
1 |
139 |
138 |
137 |
- |
- |
A |
S1 |
5,11,15,19,24,25,26,27,28,29,30,31 |
- |
Table number |
Field A from the record to be used |
Field B from the record to be used |
Field C from the record to be used |
Field D is not used |
Field E is not used |
Only A field must contain data |
Session link 1 |
Calculated fields |
No post processing fields |
Calculated fields used for IFIELDS column:
- Hits
- Validhits
- Errorhits
- Bytes
- Pages
- Nonpages
- Entrancepages
- Exitpages
- Bouncepages
- Pagetime
- Visits
- Visitors
- Newvisitors
- Priorvisitors
- Transactions
- Customers
- Newcustomers
- Priorcustomers
- Revenue
- Tax
- Shipping
- Items
- Itemrevenue
- Responses
- Impressions
- Clicks
- Cost
- Goals1
- Goals2
- Goals3
- Goals4
- Goalstarts1
- Goalstarts2
- Goalstarts3
- Goalstarts4
- Score