Monday, January 5, 2009

Quality assurance of web analytics tags implementation

This article covers the following elements:
  • The facts about web analytics quality assurance
  • The methodology usually employed to ensure quality
  • The solution: WASP v1.0 coming up!
  • Additional references: posts by fellow bloggers
  • eMetrics Toronto: round table discussion on this topic and speaking about web analytics maturity

The facts

John Lovett, from Jupiter Research, said that "42% of web analytics clients surveyed reported data accuracy as an important factor when selecting a vendor". Yet, a torough quality assurance of web analytics tags implementation is rarely part of the initial implementation and data quality is impacted as the site evolve and tags are left out of the loop.

I have personally conducted tests on dozens, if not hundred of sites, including vendor sites. So far, I have not found a single site that didn't have at least one of those common issues:
  • Missing tags
  • JavaScript errors
  • Missing data elements
  • Using the wrong account id
  • Following a semantic approach to page titles, links, site sections, etc.
  • Passing the wrong data: invalid numeric values, string with invalid characters, long strings, etc.
  • Not following the "domain of values" of some tags: for example, if you always send 1,2,3 to represent a membership levels, sending a value of "X" or "4" is an error.

Methodology

Web analytics implementation is much more complex than most people think. Quality assurance remains a cumbersome and complex task. You basically have 4 alternatives:
  1. Blind faith: Trust and wish everything is right... or slight variation on the same theme... check only the pages that are part of your KPIs and funnels and hope for the best
  2. Debuggers & proxies: Charles Proxy, ieWatch, Fiddler, Tamper Data, HTTP Watch or some vendor provided simple javascript debuggers. Visit all pages of your site to make sure the tags are working as expected. Then wait... and check in your web analytics tools of choice to see if it was recorded correctly
  3. Crawlers: Use a high-end tool such as Maxamine (now Accenture Digital Diagnostics), which is very powerful and targeted at larger companies or something like Web Link Validator, which would still require a fair amount of technical know-how. For Google Analytics there's also SiteScanGA but it also has some limitations (incomplete crawls if the site isn't fully indexed by Google, basic checking, etc.)
  4. WASP: The Web Analytics Solution Profiler was specifically built for quality assurance of web analytics implementations. The sidebar offers page-by-page view of the tags as you surf and the crawler allows you to check a full site or section of a site.

Solution: WASP v1.0 coming up!

WASP v1.0 is coming up this month and it will be very affordable. This version will include specific enhancements for Omniture SiteCatalyst and Google Analytics (such as sidebar friendlier view of tags - see screen capture) and future releases will continue to bring further enhancements.

The "Crawl from..." wizard will guide you through the steps to ease the process of crawling a whole site or a specific site section. Since the crawler runs "in context" of a real browser session, there is no better way to test. Options such as timeouts, filters, excluding your own traffic, pause/resume and data management policies makes it a unique approach to web analytics quality assurance of tagging implementations. The built-in data explorer makes it easy to spot untagged pages or wrong values being sent (see screen capture) and you always have the option the data to export to Excel and other formats.

Additional references

Some references about the challenges of tagging:

Speaking at eMetrics Toronto

I will be at the next eMetrics Marketing Optimization Summit in Toronto (March 29-April 1) where there will be round table discussions on various topics. I will be a facilitator for the discussion on "How Trustworthy is your Data?"

I will also be presenting about "Establishing your Online Analytics Maturity" in the "Jump Starting your Online Analytics Implementation: A Four Part Series". The session will be very interactive and you will get out of it with your own maturity assessment map! Get more information on the topic of web analytics maturity assessment here and here.
Labels: ,