Should you change job?

There are a number of great blog posts on starting in the web analytics field, the job market is growing and we've seen a number of high profile people doing career shifts to go freelance or start their own businesses.

You can do a deep introspection, see a career advisor, be tempted by the greener grass at the company on the other side of the street... Or ask yourself those three simple questions:

1. Am I increasing my value in the market?
2. Am I bringing the right value to my employer?
3. Am I being rightly compensated for my value?

Trough my 20 years of experience in the field, I have come to the conclusion I only needed to answer those 3 questions to decide if it was time to move on. I usually do it twice a year, a quick check of my professional situation and of my personal values. If I can't say "yes" to any of those and come up with a satisfying answer, it means there is something I should do.

Everyone can interpret the words "value" and "compensation" their own way. It really depends on your own personal values. That's why I'm avoiding to give any more details. Just think about it for a couple of minutes and you will get your own answers.

Of course, one need to balance those professional aspects with the other aspects of life: family, social, personal, leisure, health, finance, etc. But from a professional perspective, it's very straightforward.

Vista Blue Screen! Or how to make errors look good

Everybody knows the infamous Windows Blue Screen error or the simple but obvious call to action "Abort, Retry, Ignore". They both have plagued our computing lives for nearly 30 years.

One would think that with advances in software development, in the age of Web 2.0, such critical systems as Windows Vista would become legacy for those gray haired computer geeks like me. Well, I guess not!

How subtle of Microsoft engineers to remind us of a piece of legacy. In the snapshot taken this morning, there is some comfort in the sample but very efficient error message: "Problem Event Name: BlueScreen"! Almost like comfort food for the mind.



At least, now we can "Check for solution", which, of course, doesn't solve our issue but makes us feel more empowered and satisfy our inner instincts of being in control.

(Note: I tried "check for solution" without success, than canceled and Vista continued smoothly).

Web Analytics Wednesday, Sept. 12th, Montreal

After a summer break, we're back for a Web Analytics Wednesday in Montreal on September 12th! (English version below)

Web Analytics Wednesday?

Comme le veut la tradition du Web Analytics Wednesday, ces rencontres sont de natures "informelle" et il n'y a généralement pas d'agenda précis. Les discussions s'amorcent librement et vont aux grées des interactions.

Le tout se veut chaleureux, simple et une occasion idéale de rencontrer des gens qui partagent des intérêts communs autour de l'analytique et de la stratégie Web, de la conception de sites, du design, de l'ergonomie, du SEO, et même des statistiques, des aspects éthiques et légaux, etc.

Réservation

Nous sommes habituellement au moins une quinzaine de personnes, conseiller en affaires électroniques, analystes Web, gestionnaires, etc.

Chez le Portugais
4134, St-Laurent
Montréal, QC H2W 1Y8
514-849-0550

Nous aurons une salle privée à  partir de 18:00h.

RSVP: Il est important de réserver en me faisant parvenir un courriel, en laissant un commentaire ou en ligne.

Présentation/commanditaire

Si vous aimeriez présenter un service ou un produit, être un "sponsor" en offrant une tournée ou un prix de présence, ou simplement faire une petite présentation sur le sujet de l'analytique, SVP me contacter. La seule restriction est que la présentation doit viser l'éducation et la croissance professionnelle des gens, et non de promouvoir simplement votre produit ou service.

---

Web Analytics Wednesday?

The monthly meeting of the Web Analytics community is inspired by the Web Analytics Wednesday tradition, which is an happy gathering of practitioners in the field of web analytics or other related fields of interest (strategy, SEO, design, or the Internet in general).

Event details

For the month of September, we will go at Le Portugais on Wednesday, September 12th where we will have a reservation for 6:00pm. We usually gather around 15 people; consultants, web analysts, managers, etc.

Chez le Portugais
4134, St-Laurent
Montréal, QC H2W 1Y8
514-849-0550

RSVP: simply send me an email, leave a comment below or register on-line.

Speaker/sponsor

If you would like to sponsor this event, present a product or a service, pay for the first round, or simply present something on the subject of web analytics, please contact me. The only restriction, to stay in the true spirit of WaW, is that your presentation be educative or otherwise help people grow in their web analytics practice, not simply to promote your product or service.

Thinking out of the box

One of the common skill mentioned in web analytics job postings is worded as "strong problem solving". Problem solving is a skill anyone can learn, but there are some ingredients that are essential: an innovative mindset is one of them.

One of the latest ChangeThis manifesto highlights the seven pitfalls to avoid in order to break the mold of traditional thinking:

1. Shortcutting: Avoid leaping to solutions; a "blink" solution often hide deeper causes. Although most problems don't require us to analyze them very deeply, when faced with more complex challenges we have a tendency to jump to conclusions.
2. Blindspots: Experience often leads the way and can make us blind to what we are less comfortable with. We have to force ourselves to break the mold and think differently.
3. Not invented here: Sometimes it's worth to consider and trust other's solutions.
4. Satisficing: When we satisfy from a solution that will suffice... we satisfice. We compromise, instead of going for the best solution, we go for the less negative one.
5. Downgrading: Lower the bar and it's a lot easier to achieve success! Avoid overselling the upsides and shuttering the downsides...
6. Complicating: Seeing things more complex than they really are is an easy justification to add cost and time and reassure us in our lack of comfort when confronted with the unknown.
7. Stifling: Dismissing altogether, or second guessing one's idea if favor of our own.

I've worked in environments where over half of those pitfalls where commonly seen without even realizing it was happening. Younger and less formal organizations often fall for the not invented here, shortcuting while older organization are more prone to satisfice and stifle. It's insidious, and it's often part of the corporate culture.

The web analyst is often an agent of change and an evangelist within the organization. Developing an innovative mindset and problem solving skills is rarely presented in formal training, but just being aware of the pitfalls is already a good start.

Practical analytics: persuasion scenarios

A typical site is composed of several conversion objectives: registering to a newsletter, purchasing something, finding the nearest store, or any activity that represent a positive outcome for the user and the site owner. As Bryan Eisenberg puts it, those represents the "conversion points" of a "persuasion scenario":

Schema from Brian Eisenberg

Conversion Funnel

Conversion beacons represent the first and each of the steps leading to a conversion point. They are typically represented by a conversion funnel. On a typical ecommerce web site, the conversion beacons are the steps of the checkout process (a linear funnel) leading to a purchase (the conversion point).

What you should look for in the funnel:

• Distraction: how many of your visitors engage in the conversion process but leave it to wander in another area of the site? Look for next-page flow that leads to other pages of your site.
  
• Abandonment: how many of your engaged visitors are simply exiting the site? Look for the Exit Page metric for each of the steps of the funnel.
  
• Errors: anytime an error condition is presented to the user, the likelihood he/she will exit the process becomes more significant. Look for page reloads ratio and next-page flow that leads to error pages.
  

Conversion beacons

A conversion beacon is a step within the conversion process. This step is more easily analyzed with a Form Overlay tool, where metrics are shown "on top" of the form. Each conversion beacon should be analyzed for:

• User errors such as skipping a required field or using the wrong input format: eliminating errors as close to the user as possible is usually the best approach. Avoid round trips to the server for simple validations, implement AJAX calls to ease the process flow.
  
• Abandonment at the beginning of the form: a typical indication of users being overwhelmed by the complexity or the length of the task at hand. Break complex tasks in manageable chunks and clearly tell the user about where they stand in the process.
  
• Abandonment at the end of the form: a typical indication of fear of the unknown. Clearly label action buttons ("continue" or "place your order" or "confirm"?) and again, indicate where you stand in the process.
  
• Clustered abandonment: high abandonment on a specific field (or group of fields) is a clear indication of misunderstanding (what do they mean by "Enter address category"?) or mistrust (why do they ask for my birth date to purchase a book?).
• Abandonment anywhere in the form: once you've nailed down all the other aspects, there will be a number of inexplicable abandonment and there might be no rational explication behind those.
  

Points of resolution

Points of resolution are those pages that triggered the user to take action. Next in the analysis process is to find out which pages contributed the most to your conversion objective.

• Previous-page flow: when looking at the first conversion beacon of your funnel, look at the previous page flow. This will indicate where people stood before entering the process.

Waypoints

Moving up another step, we want to see how each page contributes to the conversion. What used to be path analysis as become irrelevant in most cases because the number of combinations are virtually infinite and are not significant. So we rely on the Page Value instead:

• Page Value (called "Page Conversion" in HBX or "$ Index" in Google Analytics): this is typically calculated by distributing the value of a conversion to all the pages that where viewed before the conversion. Even in a non-ecommerce site, specifying a conversion value (say, 1$ for every subscription) will be particularly useful to calculate this metric.

Funnel Points

Those are the pages that puts you in a position to influence the user toward the objective. They can be entry pages or specific pages that are specifically geared toward the persuasion process.

• Entry Pages: Look for the entry points and how they contribute to your conversion.
• Page Value: Again, the Page Value will help us identify our winning pages, but the next one might be even more interesting:
• Pathfinder: Omniture SiteCatalyst offers "Pathfinder", this will basically identify a significant subset of the click path that contains specific pages (such as "what happened between my entry page and my first conversion beacon?").
  

Driving Point

Here, the goal is to find out which "source" brought us the most qualified traffic. This topic in itself represent one of the major web analyst tasks and typically consist of analyzing the following metrics as they relate to conversions (those who engage but do not convert) and bounce rates (those who stumbled on the site and did nothing else but leave):

• Referrers: which web sites brings us the most qualified traffic?
  
• Search Engines: where do people go to find about us?
• Keywords and phrases: regardless of the search engine, what was in the mind of people that came to our site?
• Campaigns: which of our marketing activities are the most successful. Be it banners, PPC or even offline activities.

Key Performance Indicators

A very common question is "Which KPI should I track?". Here we're talking about indicators that will instantly raise a red flag (or bright green light) if they change:

• Conversion Ratio: Each of your persuasion scenario, be it a purchase, a subscription or any other valuable business activity, should be measured as a Ratio of First Conversion Beacon to Conversion Point. Have such ratios for each individual scenario. Example: Shopping Cart/Purchase, or Subscribe/Thank You.
  
• Global Conversion Ratio: Regardless of the persuasion scenario (which transaction), track how successful your visitors are at accomplishing their goals (and yours!) by dividing Any Conversion Point/Visits.

Thoughts about the Web Analytics industry

I was reading through Avinash's comments about Market Motive and started to do a parallel with an old post I did where I exposed my theory of evolution: from IBM's hardware era to Google information age. The basic questions are: What are we doing? Where are we heading?

The IT era

Logs have been available for HTTP server since day one. It was a normal thing for any network based server to have them. Gopher, FTP, email, IRC all had their logs before HTTP and HTML were popularized by Mosaic in early 1993. Log based analysis grew from the IT department and gradually and naturally propagated to marketing. Those are the golden years of WebTrends.

The Marketing era

There's always that little (sometimes bigger) tension between IT and Marketing, the Ying and the Yang of the web. One doesn't exist without the other. Yet, they speak different languages. They seek for different answers: one is about bandwidth, performance, availability, bugs while the other is about user behavior, conversion, design, usability, etc. Tagging a site and getting self-served insights was like a dream come true for Marketing. No more hard knocking with IT... at least that's what they thought! Now they had even better arguments to kick the *ss of IT 'cause "hey! How can they possibly understand it's so important to validate as close to the user as possible instead of doing round trips to the server?". Web reporting solutions sprung during the Internet Buble and the Omniture, WebSideStory, Coremetrics and a bunch of others grew from their ability to render IT dependent and time consuming reports into easily segmented, filtered and nicely shown reports and graphs.

The democratization era

Google took the world by surprise in November 2005 by bringing web analytics to everyone who wants it. Now any Joe with a blog knows what's going on and companies don't have any excuses for not looking at their numbers. It brought the tool but didn't bring the process and the resources along. The door was wide open for would-be "specialists" to jump on the new buzzword of the day: welcome to "web analytics". Companies that used web analytics with most success are those that implemented the analysis culture internally and consciously invested in resources and processes. For most, however, the tools looks all the same, "why bother and pay for something I can get for free?"... they think. Only the best will survive. ThinkMetrics threw the towel recently, and from what I'm seeing while developing WASP, there will be several other smaller players who won't be able to survive in front of the Google Behemoth.

The bubble

The web analytics industry is going trough a bubble of it's own. Innovation abound, Web 2.0 companies pops at a faster pace and acquisitions and mergers slowly but surely clean the market. Nevertheless, some of the innovative companies that revolve around web analytics, like RobotReplay, CrazyEgg, TapeFailure and others, might be lucky enough to find their niche and complement the major players.

Instead of working for a single corporation, home grown specialists seeks opportunities by becoming freelances or starting "guru" agencies. Peterson's Web Analytics Demystified, Avinash Kaushik's Market Motive are two of the most visible examples. Any serious conference now has it's web analytics track and "the only conference on web analytics you should attend" abound. Be it a Basecamp, an X-Change, continuing studies, or Vendor-X University, education becomes the holly grail of any market growing at this pace. Between all of this, the Web Analytics Association tries to "unites and fosters the interests of industry practitioners, vendors, consultants and educators who use, sell, install, implement, consult, teach or train in the field of web analytics". Not an easy goal for a non-profit organization relying on volunteers who already have their hands full.

What are we doing? Where are we heading?

The tools might be commoditized, but the ability of decision makers to act rather than react will never be. "Competing on Analytics" won't be a brilliant author vision of the future, it will be a mater of survival. "Web analytics" will leave it's "web" connotation behind and become a powerful tool in the arsenal of corporate strategies. We're not talking about "ebusiness" anymore, we're talking about a mater of fact for any serious business.

Get educated, learn from the field and borrow from the experts. Don't fall for the traditional thinking, persevere and innovate with whichever tool you have, but more importantly, innovate with your mind.

What do you think of the web analytics market? Where will we be in 2, 5 or 10 years?

Be wary of Google Analytics case studies: take two

A warning...

I posted about a little discovery of my own last week: 4 out of the 10 current Google Analytics case studies don't appear to be using it at all (at least, not anymore) and two others can mislead the reader. This prompted me to raise the hand and warn about Google Analytics case studies. I also looked at other players and noticed the market leaders have much better results while the smaller players are often worse (actually, a lot of the client list they show switched to Google Analytics).

...and a good sign

Yesterday, the Google Analytics Blog asked for your success stories. Might be a coincidence, maybe not, but nevertheless a good thing and a message to other solution providers: make sure you have an editorial timeline to review client lists and case studies posted on the web!

A little side note

Google Analytics Blog uses Blogger (I do too). Blogger, like many other blog softwares, as a feature that automatically show backlinks. So in theory, this post should actually become visible there. I wonder what will happen :)

Web Analytics implementation Quality Assurance

Here's a couple of references that should help improve the quality of your web analytics implementation:

• Web Analytics Implementation: Items Overlooked by Manoj Jasra
• Part 1: Your Web Analytics Data Quality May Stink and Here’s Why! by Judah Phillips
• Part 2: Your Web Analytics Data Quality May Stink and Here’s Why!
• Web Analytics and Data Collection: The Page Tag by Judah Phillips
• Web Analytics Technical Implementation Best Practices. (JavaScript Tags) by Avinash Kaushik

Which brings me to the next question: what would you like to see in WASP?

If you don't see the poll, please visit my blog or answer directly.

• More web analytics solutions. The current version of WASP detects 45 of them, the next one already has 50, and there's nearly 200 web analytics solutions on the market, some of them phasing out, new one popping all the time.
• Site crawl and implementation diagnostic. Missing and duplicate tags, JavaScript errors, duplicate page names, wrong location of code, etc.
• Market stats about web analytics solutions. Which solutions are leading the pack? Who's using what?
• Detailed implementation QA for a specific solution, which one? Each web analytics solution has its little tweaks and tricks.
• Others? What would you like to see?

Please fill-out the poll, send me an email or simply post a comment!

WASP 0.25 released

What is WASP?

WASP is the free Web Analytics Solution Profiler, a Firefox extension aimed at web analytics implementation specialists, web analysts and savvy web surfers who wants to understand how their behavior is being analyzed.

By automatically detecting the tags on a page and displaying detailed information about the data being sent, WASP can significantly ease the tagging process and increase the quality of a web analytics implementation.

Get more info

WASP now got a site of its own at WebAnalyticsSolutionProfiler.com where you can get more information and download the Firefox extension. And if you find this tool useful, make a donation!

What's new?

The most significant change is the number of web analytics solutions being detected: 45 of them! This version brings minor enhancements but the detection algorithm and the code were optimized. There's also a fix for that annoying problem when switching tabs or loading new pages. See the complete revision history and the FAQ. Here's the list of tools being handled by WASP v0.25:

• AuriQ
• Clicklab
• ClickTale
• ClickTracks
• CNStats
• Coremetrics
• CrazyEgg
• eTracker
• EXAConsult
• Firelick
• Google Ad
• Google Analytics
• Hitmatic
• HitsLink
• ICEtracks
• iMinr
• IndexTools
• Insite
• IntelliTracker
• Lynchpin
• Mint
• Moniforce
• MS Gatineau
• mtracking
• Nedstat
• Nielsen/Netratings
• Omniture SiteCatalyst
• OneStat
• Quantcast
• SageMetrics
• SiteClarity
• StatCounter
• Statistics.ro
• Tacoda
• TapeFailure
• Unica
• WebSideStory HBX
• Weborama
• WebtraffIQ
• WebTrends
• Xiti

Your feedback is important!

Like it? Or not? Would like to see a new feature? Found a bug? I'm building this tool out of my free time and my reward is your feedback!

If you are inclined to do so - or if you use WASP for professional purposes - a donation would be appreciated!

Some fun with a site design

Since I met Mr.Carrabis at eMetrics and attended his fabulous presentation about the human side of online marketing, I'm much more aware of how subtle design elements might affect the user behavior. So let's have some fun with a site my wife stumbled on.

What's wrong with this site?

Take a look at the picture on the right (click to see a larger snapshot). The site is actually in French, so don't worry about the wording: just focus on the design. What's your first impression? What's wrong?

Asymmetry

If your first impression was one of confusion (or maybe even disgust), you've guessed right. My 15 year old daughter wants to study in an art-related field (see one of her drawing below) and she learned that one of the most difficult thing to draw are portraits. Why? Because our brain is so good at interpreting human faces that anything that appears to be an anomaly will literally jump at our face (sic!). Humans have been doing it for millions of years to know if the person in front of us was hostile or friendly. You've probably heard about the little experiment where you take a person's picture and use Photoshop to create a half mirror representation of the face. We instantly detects there's something wrong.

Why do this? What could be done instead?

This company is about human resources. I don't know anyone there and don't have a clue about who designed it. My only guesses for such a choice are: a) represent the two sides of the job hunting process: employer/employee. b) be politically correct by showing mens and womens "at the same level". Furthermore, the animation makes us focus too much on the pictures instead of the left and right-side navigation.

There's a nice section where employers and employees gives short testimonials. Wouldn't it be great if their picture was taken and the blurb be put right beside it on the home page? A single display (maybe chosen at random) and previous/back buttons would be less intrusive than an automated animation. I would love to do A/B testing on that and see the results!

VisualSciences HBX Active Viewing Form Overlay

Someone asked on the Yahoo! Web Analytics forum about HBX Active Viewing form overlay functionality.

My experience

• The nicest thing about HBX is that whenever there are more than a couple of fields on a page, form analysis is automatically turned on. No fancy tagging, no after-the-fact missed tags, etc.
• The Form Overlay works just like the "plain" Active Viewing overlay. It will show each form field with a color tone representing it's impact on conversion, along with a bunch of useful info
• You might have trouble installing the plug-in on Vista because it requires .Net 1.1... In fact, Vista is not in the list of supported operating systems... So the latest version of Active Viewing doesn't work, but here's a hint: search for Active Viewing in their support center and you will find a link to an older version that works :)
  

Does it really work?

Absolutely! Really easy to use and quite revealing. I ran into a couple of cases where the overlay didn't work and it usually turned out to be badly built HTML pages: nested forms, invalid HTML syntax, or uses of JavaScript to alter the normal behavior of a form.

Also, if the site uses a lot of CSS layering, you might end up with the overlay behind your form fields, which is annoying but you will be able to get the results anyway.

Analysis

Here's a couple of things you might find out with the Form Overlay:

• Lots of abandonment on the first field? Might be an indication that people get scared of what they have to fill!
• Lots of abandonment on the last field? Might be an indication that people don't want to commit to the transaction (for example, in a shopping cart process, a lot of people abandon once they get the total price)
• Lots of abandonment on a specific field? Might be a badly worded question or out of context question (asking for gender to purchase a CD!)
• Gradual abandonment as you go own the list of fields? Might be an indication you have too many fields, people gradually drop off by discouragement.
  

My take

HBX's Active Viewing is by far the best I've seen, providing detailed information as you surf and the overlay feature works under almost any kind of situation: plain HTML, dynamic menus, image maps, even dynamic links, etc. (in some cases, more extensive tagging needs to be done). Although you can get similar information from the regular reports, and especially the Form Abandonment report, there's nothing better than showing the numbers "in context".

Omniture Discover 2.1: just like playing Lego

Back from vacation, I was eager to see if our Discover v2.1 license had been turned on. Sure thing, it was there, boldly saying "Launch Discover 2.1".

What's changed?

• Licensing: At eMetrics I heard some people complaining they had to pay for the D2 upgrade even if they were already licensed for D1.5. In fact, D1.5 was based on a per-user/per-reportsuite model, while the new licensing is per-seat and for all your report-suites. So unless you only have one analyst and one report suite, the new licensing model is much more interesting.
• Interface: The interface is totally changed and enhanced with new visualizations, including the traditional pies and bars, but also funnels and the amazing 3D site analysis. Hey! Omniture too can do fancy little gadgets like a time-slicer to easily pick date ranges just like Google Analytics :)
• Reports/Segments/Metrics/Comparisons: the intuitive interface is straightforward:
  
1. Pick a report: the same from SiteCatalyst, including all of your custom vars appropriately named
2. Chose a segment: including pre-configured ones and those defined in SiteCatalyst, or define new ones using the Segment Builder
3. Pick some additional metrics, including custom ones or define new ones trough the Metrics Builder
4. Compare all of those with different segments or time frames
   
• Slicing & dicing: playing with the data, drilling down (nested metrics), filtering or arranging the columns is a snap, plus the graphs gets updated dynamically and is easy to tweak (like pulling down a slice from a pie chart).
• Other cool features: site analysis is the amazing 3D visualization of your site navigation along with the metrics of your choice, dynamic path flow lets you navigate the site and drill down in any way you want and the virtual focus group (a feature similar to ClickTale, TapeFailure and such playback tools) are just way cool!
  

First launch

After firring up Discover v2.1 trough Java Web Start and a couple of quick downloads, the new interface came up. A drastic change from v1.5! Here's some caveats to watch for if you have difficulties starting it:

• Although the latest version of the required Java runtime is JRE 6 Update 2, make sure you are using JRE 5.11. If you don't, Discover will warn there might be some problems... and there are. See KB #1870.
• If you have Google Web Accelerator installed, disable it while using Discover. Otherwise you will end up with the Java Web Start window looping continuously, trying to download the files from Omniture. Also refer to KB #1276.
  

Advanced analysis capabilities: real stories from the trenches

Like Craig Danuloff reported a couple of months ago when he first saw D2, SiteCatalyst "as a reporting plus a little analysis and the other (D2) as a hard-core analysis tool". Here's two examples (a bit simplified) of things I couldn't do with SiteCatalyst:

• Complex funnel analysis: Imagine you have a car insurance quoter where you define the driver, the location, the car, and get a price (4 steps funnel). We have several transactions that share some of the steps (like "Add a driver" to an existing insurance policy), and some steps that have multiple alternative versions (like getting a quote for two cars or two drivers at once). SiteCatalyst allows you to build a funnel but each step must be a unique page, which made it very complex (even impossible) to address all our complex needs. With D2 page groups I was able to easily solve this problem, plus I can now dynamically change my segmenting (dynamic segmentation is a feature that WebSideStory HBX Analytics is very strong at).
• Nested metrics: doing "as you go" analysis is often much more efficient then having to extract a bunch of reports and play with them in Excel pivot tables... I wanted to validate if a geotargeting feature would be efficient on our site. Using the nested metrics capabilities, I was able to easily drill down, switch segments and see if what the automated targeting would have chosen would effectively be what the real user ended up picking (comparing GeoSegmentation and a custom variable holding the real user's selected region). For example, the system think you are in Quebec, and you really are: great! The system think you are from Ontario, but you end up doing a quote saying you live in Quebec because Gatineau,QC and Ottawa, ON are just beside each other, but in two different provinces (check the map!). Are we comfortable with that? In the end, I was able to conclude the solution would be accurate 99% of the time in some regions, and would be very poor in others. Knowing that gives us actionable data and we can take the right business decision. (I lived a similar situation in a previous job, where a dealer locator feature would automatically recommend the nearest dealer - on the other side of a river that required a two hour drive to get across a bridge! What was the ratio of false-positive? What was the consequences on conversion rates? We never knew because we didn't have any web analytics tools).
• For another real-life example, see A case about reporting vs. analysis

My take

In my view, once you get beyond reporting, once you've reached a certain level of analysis capabilities and brought the web analytics culture to your organization, the next logical step is to use powerful tools like Discover. But one thing remain: out of the critical activities of a web analyst, exploring new strategies is one of the most difficult and sadly, the one we have the less time to do. Discover is a really good product, but it won't do any magic: better make sure you have the right people, the right process and enough time to really use it as it should.

Be wary of Google Analytics case studies

Is it ethical to keep online case studies of clients that are not using your solution anymore? Or to present case studies in a way that could mislead your potential clients?

What I found out

I'm getting closer to the next version of my WASP Firefox extension, and to ease the testing process, I figured I could build a quality assurance test where I would visit client sites referred by each vendor. A list of high-profile clients shown with emphasis on the home page and case studies is a proven way to increase confidence and credibility. But here's what I found out...

Google Analytics case studies

I was in for a surprise! Take the case of Google, which highlights a number of case studies:

• BuildDirect Google Analytics
• Discount Tire: Google Analytics
• RE/MAX: Google Analytics
• Roche: Google Analytics
• Vueling: Google Analytics
• CKE Restaurants, for the Spicy Paris campaign: Google Analytics
• Financial Times: WebTrends live, Google Analytics is used for campaign landing pages
• Agency.com: WebSideStory HBX v1.4... no traces of Google Analytics
• Career Builder: Omniture SiteCatalyst, even CBCampus.com, the site referred in the case study, uses Omniture and as no traces of Google.
  
• Deckers: none identified (might be log based).

Could it be that 4 clients out of of the 10 Google Analytics case studies are not even using it? They might have been in the past, they are not anymore? Saying the Financial Times uses Google Analytics is a sure way to bring credibility, but when digging the case study, we find out GA is used for landing pages only. While the core site, FT.com, relies on WebTrends.

I'm sure Google Analytics is not the only solution provider in this situation. In fact, I've even found a number of vendors that are not using their own solution on their corporate web site...

What about others?

I took a look at a couple of the Omniture Customer Showcase and a random sample of 10 of those sites resulted in 100% Omniture usage (along with some double-tagging with Google Analytics). Same thing for VisualSciences WebSideStory Customers... I can stop there.

My two-cents

1. Can you put a date on the case studies? Being in the web analytics field, you know that what makes a good story is the context. Without any time reference, we loose a valuable contextual element that can easily lead to assumptions and misjudgment.
2. Take their claims with a grain of salt (and pepper). Everybody is a "market leader", and an "industry first" at something. Cut the crap and do your homework: does the solution fit your needs, not those of others!
   
3. As they say at Microsoft: "Eat your own dog found": if a vendor is not even using its own tool (and especially if using Google Analytics!), how can you believe their solution is anything but useless? (This is especially true of smaller players, but even some of the major vendors are using both their own and Google Analytics!).

(shameless plug)
WASP can help you detect which web analytics solution is being used by top sites and competitors. And it can help you insure the quality of your own site tagging.

WASP got a site of its own

In preparation of the next release of WASP, the Web Analytics Solution Profiler, I have decided to build a site just for it. I think it has enough of the bells & whistles of the Web 2.0 style to be cool :)

If you could head to WebAnalyticsSolutionProfiler.com and share your impressions and critics (via email or comment below), that would be appreciated!

One thing that will be done shortly: include new snapshots or even a short Flash movie of WASP in action.