What is A/B Testing and how can it help me?
A/B Testing allows you to compare the effectiveness of different versions of advertising and other content at referring quality leads and customers. Often, multiple versions of promotional content link to the same landing page on a web site. Content (A/B) Testing provides a way for you to differentiate each version of content, even when all versions of content link to the same landing page, so that you can see which versions are most effective. Urchin Campaign Tracking reports click through rates, new leads, average page depth, loyalty, conversion, and revenue for each version of content, for any specified source (such as newsletter title).
What are the steps to performing A/B Testing?
- License and Install Urchin
You will need to purchase and install the Urchin base product and the Campaign Tracking Module. If you need keyword ROI metrics, you should license the Profit Suite, which includes the Urchin base product, the Campaign Tracking Module, and the E-Commerce Module. Read Step 2:Install and License Campaign Tracking for more information. - Define a Conversion Goal
Read Step 3:Define a Conversion Goal to learn how to specify a goal for your site. - Set Up Referral Links
Usually, you will want to see how different advertisements perform in generating referrals and business to your site. Each link will have different kinds of text and graphics. For each of these ads, you will need to set up a referral link to your site and embed UTM variables. This article describes the best way to use the UTM variables for A/B testing, below. - Track Campaign Data
You will need to install the UTM-3 or UTM-4, enable cookie logging, and embed UTM variables in your referral links. The article Step 1:Track Campaign Data contains information on how to do all three of these things, and additional information on how to use the UTM3 variables for keyword buying analysis is provided in this article, below. - Import E-commerce Data (only necessary for e-commerce ROI analysis reports)
- Optimize Your Ad Content
Information on how to use the Content (A/B) Testing reports to optimize your ad content is provided in this article, below.
Which UTM variables should I use for A/B Testing?
For A/B testing, you should use utm_source, utm_medium, utm_content. If you have multiple types of products or multiple campaigns, you should also use utm_campaign. For example, if you have a "spring sale" campaign and an "adidas" promotion, you should indicate the appropriate campaign in your link.
The following links illustrate how you would use the UTM variables in an email newsletter that contains multiple links to a conversion page on your web site. Although both links refer the user to the same page, the first link is for a text:
| Example: referral for users that click on the text |
|---|
| http:// www.mycompany.com/buy_page?utm_source=newsletter&utm_medium=email&utm_content=buytext |
The second link is for a graphic:
| Example: referral for users that click on the graphic |
|---|
| http:// www.mycompany.com/buy_page?utm_source=newsletter&utm_medium=email&utm_content=buypict |
Urchin provides a URL builder tool that creates links for you, embedding the campaign information that you specify. Using this tool ensures that your links contain the correct syntax.
Be consistent with UTM variables.
It is important that you use consistent names and spellings for all of your campaign variable values. For example, choose a code or name that indicates "cost-per-click" and use it consistently. To Urchin Campaign Tracking, utm_medium=cpc and utm_medium=cost_per_click are different mediums.
Beginning with Urchin 5.6, a master tracking code feature is available that significantly reduces the possibility of consistency errors. Read How To Use Master Tracking Codes.
How to use the Content (A/B) Testing reports to optimize your advertising content
- Which versions of my advertisements are most effective at driving leads to my site?
- Which versions of my advertisements refer the visitors most interested in my site?
- Which versions of my advertisements refer the visitors most likely to reach a conversion goal on my site?
- Which versions of my advertisements provide the biggest return?
Which versions of my advertisements are most effective at driving leads to my site?
Look at the Content (A/B) Testing-->Acquisition report. In the Filter field, type "referral|none|organic" and click the minus button. The report now displays only referrals that were tagged with UTM variables. For example, in the following report, the report displays referrals from three source[medium] combinations.
Drill down on the source[medium] for the content you would like to examine. The report now displays the different versions of content within that source [medium]. The CTR (Click Through Rate) tells you the percentage of impressions (ad displays) that resulted in clicks. The %New field tells you the percentage of clicks that are new leads.
Which versions of my advertisements refer the visitors most interested in my site?
Look at the Content (A/B) Testing-->Quality report. In the Filter field, type "referral|none|organic" and click the minus button. The report now displays only referrals that were tagged with UTM variables.
Drill down on a source[medium] for the content you would like to examine. The report now displays the different versions of content within that source [medium]. Depth is a measure of interest which tells you the average number of pages on your site that each visitor viewed. Loyalty indicates the average number of times visitors returned to your site.
Which versions of my advertisements refer the visitors most likely to reach a conversion goal on my site?
Look at the Content (A/B) Testing-->Conversion report. In the Filter field, type "referral|none|organic" and click the minus button. The report now displays only referrals that were tagged with UTM variables.
Drill down on a source[medium] for the content you would like to examine. The report now displays the different versions of content within that source [medium]. Goal Conv. (Goal Conversion Rate) is the percentage of referrals that reached a goal on your site. If you have licensed the the E-commerce Module, the percentage of referrals that made a purchase on your site, Sales Conv. (Sales Conversion Rate), is displayed.
Which versions of my advertisements provide the biggest return?
If you have licensed the E-commerce module, you will be able to see the revenue associated with each version of content. Look at the Content (A/B) Testing-->ROI report. In the Filter field, type "referral|none|organic" and click the minus button. The report now displays only referrals that were tagged with UTM variables.
Drill down on a source[medium] for the content you would like to examine. The report now displays the different versions of content within that source [medium]. Revenue is the gross revenue associated with referrals from the content within the source[medium]. For Content Analysis, the Cost column will always be 0 and the ROI column will be equal to Revenue.