Web Analytics Maturity Model: requirement gathering & risk assessment tool

Following yesterday's post, I got an email saying "Your model is very interesting as it helps visualize where any company stands and where it should go (and focus effort). This could help me illustrate the gap between our current situation and an "ideal" one. I was wondering if you could share some more insights about it"

Criterias & meaning of each score

Part of the challenge was to come up with "normalized grades" so the criteria for each level of each dimension are well calibrated. And it seems to be making sense so far.

I’m using this expertise as a value-add for my clients and a potential source of consulting revenue, so I'm not really open to share the underlying rules for now. But I can share another example to illustrate the concept.

Fit gap & risk assessment

In yesterday's post, I had included a graph showing a real client who had really unrealistic expectations. This other, more positive example, is form a client I met recently. The stated project requirement was simply to move from a log-based solution to the tagged version of the same tool. The assessment revealed a good and fairly well balanced web analytics maturity. We used the model as a business requirement information gathering and project risk assessment tool.

The graph bellow was part of a ten pages analysis report:

Doing the exercise, we clearly demonstrated the project wasn’t “only” about implementing tags. If they had been pushing in all directions, or seeking to move more than 1 grade level into a direction, it would have been a warning flag. Jumping ahead two grades is usually a sign of high risk!

Service offering

The Web Analytics Maturity Evaluation (WAME) process involves:

• Presentation of the model and information gathering
• Additional research on my side
• Writing a customized report
• Presentation of the results
• Adjustments & clarifications

If you are interested in getting your own evaluation, please contact me.

Web Analytics Maturity Model and Critical Success Factors

Update: White paper, self-assessment & more at immeria.net/oamm

During the Industry Insight day at last month eMetrics in Washington we did a round table where we were tasked by Jim Sterne to answer three key questions:

• What's the hard part?
• What do we need from the tools?
• How do we convince upper management?
  

Since I already took position regarding the concept of "web analytics is hard", I figured I should look at something else...

We got interested in "showing a long term goal and a map", basically, "use a maturity model to show what we can achieve". I had already used the excellent maturity model from Bill Gassman, of Gartner (click picture thumbnail) as a starting point with my clients. But speaking with Bill himself, and a few others, I came to the conclusion there could be a model looking further than the level of use of the analytical tools.

Critical Success Factors

What I want to develop is a web analytics maturity model encompassing the most important factors of success, basically, the Critical Success Factors of web analytics.

Wikipedia defines a CSF as:

Critical Success Factor is a business term for an element which is necessary for an organization or project to achieve its mission. They are the critical factors or activities required for ensuring the success your business. The term was initially used in the world of data analysis, and business analysis. For example, a CSF for a successful Information Technology (IT) project is user involvement.

Isn't it nice? A close relationship between data analysis and business success! While we focus a lot on KPIs, I think I have never heard anyone talking about CSF in web analytics!

The concept of CSF is also close to SixSigma, a methodology for continuous business process improvement by eliminating defects, and a strategy I recommend for web analytics.

Web Analytics Critical Success Factors

As a web analytic advocate, I get asked about Google Analytics vs other tools all the time. I kept on saying "it's not about the tool, it's about the craftman", but that wasn't enough. I think the tool should really be the last of our worries.

So now, I use a Web Analytics Maturity Model of my own to evaluate the current "state of the union" and manage more realistic client expactations. And the results are shocking!

It's a work in progress, but I have so far identified six critical elements of success, each scaled from 0 to 5:

1. Management: do you have a web analytics champion? At which level within the company?
   
2. Ressources: are you staffed correctly? Do they have the right background and training?
   
3. Objectives: can you define SMART web analytics objectives?
   
4. Scope: do you aim to do web analytics to measure the marketing performance of a micro-site or reengineer your business processes to optimize your online business?
   
5. Process and methodology: do you improvise or use a proven methodology for continuous testing, learning and improvement?
   
6. Tools and their use: now it's time to talk about wich tool is the best... for this specific situation!

I have defined a set of criterias for each dimension and scale. I'm actually working on developing recommendations for each scenario.

Using the model

Tell me, where is web analytics most succesful?
The first one is theory, the second one is a real case!

Out of the initial evaluation, I can quickly identify:

• if the current web analytics culture is at risk
• if the goals (the reason the client called me in the first place!) is realistic
• recommendations that goes beyond selecting a tool and implementing it...
• (if I really want to work with them!)

The next steps

First and foremost, as a web analytics advocate interested in the field of education & research, it's an area I want to explore further. In the coming months I will evaluate each of my clients and others who are interested. I will also present this topic at eMetrics Toronto, on March 29th, and a longer version at eMetrics Sam Jose on May 7th.

To finance this study, I'm also offering to evaluate your own business and deliver a comprehensive analysis & recommendations: contact me if you are interested.

EpikOne Google Analytics Seminars for Success coming to Montreal

Got a note from EpikOne: their popular Google Analytics Seminars for Success is coming to Montreal, December 8th & 9th at Le Centre Sheraton.

The seminar will be led by Justin Cutroni, Director of Web Analytics and has been developed in conjunction with Google.

The seminar covers the basic setup but also advanced visitor segmentation and keyword optimization, including new GA Enterprise features. The first day covers introduction & user training, while the second day looks into advanced features and technical implementation ($498 USD for both days).

Why do we start to hate Google for building fences around us?

My post entitled "Some controversy in the field of web analytics... at last!" raised interest, and it was exactly my intent when I wrote it. Without discussion and debates, we are just a bunch of nerds all agreeing on the topics that we are often the only ones to understand (we must admit "web analytics" is a nerd field... isn't it?)

One of my statements got the attention of Brian Clifton, author of "Advanced Web Metrics with Google Analytics" and a former Google UK employee:

I haven't been following all the Yahoo forum threads, so out of interest, what you mean by "we start to hate you [Google] for building fences around us, then slowly turning them into cages walls."

Over the years, people began to hate Microsoft because of its sheer power in the marketplace. Slowly, Google power is growing and the number of people who, for good or bad reasons, will tag Google as "bad", "wrong" or maybe even "evil" will increase. I think this is unavoidable.

Regarding web analytics, on one end Google is helping the industry by raising awareness and bringing free features to the mass, on the other side, it will gradually kill lots of players in doing so. I'm sure they won't really love Google for that... but it's also the hard laws of the market (or is it? Isn't it becoming unfair advantage to offer so many things for free? Some would certainly say so...)

As we move so many of our "stuff" toward Google we become vulnerable to the will of God (or Google, as you wish)... Here's what I currently use from Google:

• Google Mail
• Google Calendar
• Google Docs
• Youtube videos
• AdWords & AdSense
• Google Analytics
• Google is primary source for search
• Google News, including world news, local news, economy, companies, products, etc.
• Google Finance
  
• Google Trends
• Google Reader
• Google Forum
• Google Maps
  
• Google Apps for my domain
  
• even this blog is using Blogger
• I might be forgetting a couple...
  
• who knows what's next...

All of that for free? With no master plan other than "organize the world's information and make it universally accessible and useful."? Just "make money without doing evil"? In history, have we ever seen such a concentration of power not derail at some point?

I'm sure most of you have seen the video Epic 2014 and 2015, or "Googlezone"... it already dates a couple of years, but the essence of the message is even stronger now than it was 5 years ago.



Am I the black sheep for being somewhat scared of the concentration of power?

WASP v0.73 released

Making my way toward WASP v1.0, here's another interim release, WASP v0.73.

What is WASP?

WASP is the Web Analytics Solution Profiler, a specialized Firefox extension aimed at web analytics specialists who wants to do quality assurance and understand how their web analytics solution is implemented.

WASP personas:

1. Convenient sidebar for Web Analysts
2. Powerful site auditing and crawling feature for implementation specialists
3. Market Research features for vendors, market and financial analysts
   

Changelog for v0.73

General improvements:

• New tool detected: ClickDensity, for a total of 125 tools!
• Site moved from http://wasp.immeria.net to its own http://WebAnalyticsSolutionProfiler.com
• Released a short video showcasing how WASP Market Research feature works

Sidebar:

• New! copy values in current tab to clipboard
  
• Enhanced: show Google Analytics cookie details as 1st party cookies
• Enhanced: added menu to sidebar for easier access to all features
• Enhanced: cosmetic changes to the About tab

Site audit/crawl:

• Enhanced crawl: option to disable loading of images, CSS and Flash when crawling, for faster processing
• Bug fix: database set & create were improperly configured
• Bug fix: now correctly accept short host names such as http://localhost as crawl starting point
• Bug fix: won't hang on single page crawls
• Bug fix: database field disabled to avoid bad references (i.e. must use Open or Create buttons)
• Bug fix: fixed element help when no help is available
  

Reporting & export:

• Enhanced: faster processing of built-in data browser
• Bug fix: fixed page browsing, sorting, column filtering & reordering
• New: export to Google News sitemap.xml format
  

Get it now!

Visit WebAnalyticsSolutionProfiler.com and click on the "Add to Firefox" green button.

Some controversy in the field of web analytics... at last!

When a group of people all agree and nod their heads to any opinion that is theirs, "wisdom of the crowd" sometimes becomes "stupidity of the mass"... No surprise, the web analytics industry follows the same pattern. Let's look at three hot topics:

1. The economy
2. The (free) holly grail of web analytics
3. Hiring: dumb & dumber
   

1. The economy

We comforted ourselves in saying "when the economy is hard, web analytics is even more important". As a group, we all agreed with big "YES!" head signs. It seems we just forgot to tell everyone else outside our group! In this economical tsunami, our industry is getting hit like any others. Vendors sales are going to dramatically slow down, agencies are going to see some of their tasks be relegated to a later time or simply canceled, and analysts are likely to suffer unless their employer is already deeply involved in analytics and is already getting a big ROI on it.

2. The (free) holly grail of web analytics

Google... Google... as much as we love your free love, we start to hate you for building fences around us, then slowly turning them into cages walls. As a herd, we look at the great pasture of free stuff and we all move into that direction.No abyss to cross, just easy stuff, the path is straight and we just have to follow it.

Then there are some black sheep troublemakers who try to tell us it's a trap. It can't be true! There's always a cost! Nothing is free! And they are right, because once we are all nicely parked within the boundaries set by Google, grass will eventually become less tasteful and so scarce we won't be able to survive unless we eat each other.

As much as Google Analytics is great and raise the bar, I don't think it's the magic solution some people would like us to believe. Don't get me wrong, GA is great in some cases, not all. There is still lots of room for other players, but the playing field is getting smaller as Google gradually grab a large chunk of the prairie for itself. We haven't seen such a great thread on the Yahoo! forum in a long time. Like other things in life, what worries me are the uneducated, almost extremist believers who have never been exposed to other web analytics solutions, large projects and business cultural changes and who are becoming experts in the field after "discovering" web development, online marketing, ebusiness strategies and business intelligence web analytics just six months ago.

3. Hiring: dumb & dumber

Another great thread on the Yahoo! forum: Web analytics hiring questions.

• Rule #1: diploma is no proof of intelligence: Some of the replies reminds me of the days when I had a college degree and was told by some potential employers I was too stupid for them because I didn't have a graduate degree (not how they said it... how I felt!). In highschool I had poor grades, in college I was among the top students. Years later, I've almost completed my MBA and so far I've been on the honor roll twice for being among the 5% top students.
  
• Rule #2: admit you don't know everything: I once had a full day round of interviews with a bunch of tests like some people sugested in the thread. Clearly abusive to spend a whole day doing that... Anyway,  by the end of the day I met the CFO, who asked me a very specific question about finance I couldn't answer. I did my best to offer a logical answer and clearly said if I had the job I would look for the answer and get back to him. Thus, I admitted I don't know everything... that's the single thing that failed during that day and the HR person was really sorry to tell me I didn't get the job because of that... looking back at this, I'm glad I didn't get it!
• Rule #3: broader experience might be better: web analytics is a very young field of expertise, taking from traditional marketing, statistics, information technology, management and a whole bunch of others. As such, I think it's better to demonstrate a strong sense of autonomy, desire to learn, leadership, analytical mindset, communication skills and political acumen rather than acute knowledge of a tool such as SAS or any specific vendor, or concepts such SQL and Web 2.0.

Anecdote: Can you believe in my early days as a programmer (20 years ago!) an employer asked me to cite all the keys on a keyboard, in order please! Asking a specific SQL sentence is the same type of stupid question - and yes, I can answer that later one, but I still can't recite all the keys on my keyboard!

I would rather look for someone who can take an ebusiness challenges, identify the problem, find out the underlying issues, understand and/or define the objective and goals, how they would measure success and what kind of recommendation they would make.

Debate is open! Pick you topic and fire at will :)

WebCom - Nov. 12th, Montréal

For those interested, WebCom is coming up on November 12th, in Montréal. The conference program is built around three tracks: marketing, communication and technology with a slant on social media. It's always interesting to hear local speakers, because they are often closer to our realities, and at the same time, the international speakers often bring views that stretches the envelope.

Among the sponsors there are local web agencies and large corporations, including the French web analytics vendor who recently opened offices in Montréal, AT Internet (formerly Xiti). In pure French tradition, Dennis Varganyi, Country Manager for Canada, told me they will be giving away a vintage bottle of Bordeaux wine :)

Note that Web Analytics Association members can get a 15% discount!

I had the chance to be a speaker at a previous one, this time my schedule doesn't permit me to attend, but I look forward to hear about it!

WASP Market Research: behind the scene of the Top 500 Retail Sites study

In my previous post, I shared my PowerPoint presentation from the recent eMetrics Industry Insight in Washington. This time, I want to highlight how the WASP Market Research feature was used to conduct this study.



You can download the input file here.
You can get the raw CSV results here.

Analysis

WASP expects a list of URLs as input, the WASP Market Research feature automatically takes care of resolving the company names to their corresponding URLs. Once the analysis is complete, you can use the built-in data browser or export to CSV. In this case, using Excel sorting, filtering, Pivot tables, it's easy to spot:

• The type of tools found on those sites
• Which vendors are the most popular
• Which tools are the most popular
• What is the level of "double tagging" between various tools
• "Affinities" between various products such as Voice of Customer, Behavioral Targetting, Multivariate Testing, etc.
• Possible relation between the type of tool and the financial results (this data would have to come from a different source)
• etc.
  

Additional information

WASP Market Research is a service included in the Professional licence of WASP, currently available to vendors, agencies and market analysts. I'm also offering confidential/custom market research data as part of my consulting practice. Contact me or visit WebAnalyticsSolutionProfiler.com for more information.