Thursday, June 24, 2010

Sherlock Holmes and monkey mimicking

Most so called "web analysts" are monkeys mimicking others without understanding what they do!

In my experience teaching at UBC and ULaval, the first message a student post in a forum; the first assignment delivered, tells me if this student has the potential to become a good analyst or not. Does he or she show a structured approach to problem solving? Does he/she go beyond the obvious?

This is referred as what Jim Novo brilliantly exposed as "the Sherlock Holmes problem" - and incidentally what is being tested by the WAA Certification: go for the facts and come up with the most optimal and realistic conclusion/recommendation under the current business conditions.
"A client is to me a mere unit - a factor in a problem."
Sherlock Holmes

Evidence I: broad and useless

At the eMetrics Marketing Optimization Summit in France, one of the presentations felt for this common error: reporting page views and visits rather than bringing insight on business outcomes based on each of the customer lifecycle stages. Without being too harsh, I would say it's a start... but it is far preferable to have more depth than breath. Focus on a narrower area you can control and make a difference rather than report widely on metrics that you don't control and won't result in any business change.

Evidence II: pulling the wrong lever

The web analytics course I created for ULaval stems from my work on the Online Analytics Maturity Model and some of it is covered in my workshop. The twelve weeks master-level course includes a short assignment every week. At the earliest stage, when asked to do an analysis of the visits metric most will go in the tool, copy the graph and textually write "the number of visits was X and the average was Y"... Later, when faced with the challenge of recommendations, they will seek to improve the visits metric... fine, but how does this contribute to actual business objectives and outcomes?
Case study: Students were asked to do an analysis of the Visits to SaveTheChildren.org in January 2010, just when the Haiti earthquake stroke. Most students did not mention the traffic was 14 to 15 times higher than usual. Some did not realize the statistical shock was a direct result of increased awareness of Save The Children resulting from the disaster.

Evidence III: chasing the monkey

Most analysts are feeding from "Top 10 tricks to improve SEO", "Top 5 social media tactics", "Top 3 wonders of the online world"... Good for the "now" part of learning, and maybe a necessary step, but in the long run it's not how you will transform the business. This is what I covered in a previous post entitled "Undermining our future as web analysts".

Elementary my dear Watson

I think the starting clue is knowing one's analytics maturity. In the latest Sherlock Holmes movie incarnation, the hero thinks about his moves before going into a fight. He analyzes the weak spots and the actions that will result in the desired outcome. It doesn't only make up for a great movie and special effects, it also highlights the strong analytical capabilities and result driven approach of Holmes!