The Human Metric of Web Analytics

This article was recently featured on iMediaConnection. It is reproduced here and slightly enhanced.

Integrated marketing guru Don E. Schultz coined the phrase "marketing is static, but the consumer is dynamic" nearly two years ago. Businesses must rely on the expertise of their web analysts to tap into the invaluable data collected through their websites.

If anyone still has doubt about a career in web analytics, the recent eMetrics Summit revealed just how hot the job market is. Attendees and exhibitors looking to hire wore a green dot on their badges, a simple and effective method that clearly demonstrated the huge demand for web analysts.

"As an experiment, it was too successful," says eMetrics Summit producer Jim Sterne. "We don't want to scare away managers who are worried their employees will be poached at the next one in October, so this was one time only."

Furthermore, results of a study by WebAnalyticsDemystified revealed that nearly 50 percent of those already in the web analytics field are considering looking for a new position. The Web Analytics Association job board lists dozens of available positions and Indeed.com, a job hunting aggregator, shows a 400 percent increase over the last two years for jobs containing the term "web analytics."

What should companies seeking web analysts be looking for?

1. Experience: web analytics is not an entry-level position
2. Multi-disciplinary background or training
3. Attention to detail and ability to focus

Experience

Despite being in web analytics for fewer than five years (77 percent), and most of them fewer than 3 years (52 percent), nearly 60 percent of web analytics professionals consider their job to be "difficult." It appears that most web analysts didn't land in their current job on their first assignment; most of them have grown from other related fields.

Paul Holstein, project manager at CSC Financial, reviews the situation: "We searched for an analytics analyst for more than six months and finally gave up looking for an experienced person. We hired a bright and motivated analyst who we could train in web analytics. We benchmarked what attributes we were looking for and began our search for a curious, intelligent, driven sort of person."

Surprisingly, fewer than 30 percent of companies have a dedicated resource for managing web analytics and more than 40 percent recognize they are not maximizing the potential of their current web analytics solution (source: Aberdeen Group).

Multi-Disciplinary Background or Training

Web analyst expertise is multi-dimensional. Knowledge of interactive marketing and web design/usability, statistics, web technologies and internet concepts, as well as acute analysis and communication skills are just some of the competencies of the perfect web analyst. While most web analysts struggle to satisfy very diversified expectations, those who have the chance to be part of a multi-disciplinary team with the maturity of an analytical process and culture will benefit the most.

But what are web analysts really doing? What makes this job so difficult? A few months ago, web analytics guru Avinash Kaushik proposed that web analysts should spend their time on five major activities:

1. Reporting, providing results from various metrics to find out about the "what and how much";
2. Analyzing acquisition strategies and basically everything that brings traffic to the site;
3. Understanding the onsite customer experience and if the "persuasion scenarios" are working as expected;
4. Plugging into the business context, keeping up with the operational and strategic changes that might affect the web;
5. Exploring new strategic options, experimenting and improving the site, keeping up with the web and analytics evolution.

Attention to Detail and Ability to Focus

Taking for granted "attention", the cognitive process of selectively concentrating on one aspect of work, is the scarcest resource of all, each task was evaluated by mapping the attention type required to accomplish it. According to the AttentionScape methodology developed by Davenport & Beck in "The Attention Economy", the most effective attention will go to items that are shown near the center of the chart. The detailed survey of 36 web analytics practitioners mapped each task under three different dimensions:

1. Front-of-mind vs. back-of-mind: The attention type of the music you listen to while working is back-of-mind, in that it doesn't involve a focused attention. Front-of-mind attention is conscious, focused and explicit.
2. Voluntary vs. captive: There are some activities you can't avoid, and some others you deliberately choose to do because you want to learn or because you enjoy doing them.
3. Attractive or aversive: Some tasks bring positive experiences, while we have to do others to avoid negative consequences.

The study revealed that all web analytics activities require a relatively high level of front-of-mind attention: concentration and focus is required. However, too many front-of-mind activities can lead to anxiety, stress and a feeling of being overwhelmed. Speaking with some analysts revealed that this is a common feeling among practitioners.

Where to go from here?

Some people might say the results of the study are not surprising, but in a way it is comforting to see that between the goal and the current state of affairs, web analysts live with similar challenges and struggle with the same hurdles. There is still a lot to do in order to bring the most valuable strategic insights to organizations deploying web analytics. The tools themselves are already providing a lot more functionality and information than anyone can handle, but as more educational resources become available and businesses come to realize the value of web analytics, the role of web analyst is likely to become critical not only to a successful internet presence, but maybe even to the success of entire organizations.

The role of web analyst is critical to a successful internet presence and maybe even the success of entire organizations.

As Davenport, Cohen & Jacobson say in "Competing on Analytics", "Employees hired for their expertise with numbers or trained to recognize their importance are armed with the best evidence and the best quantitative tools. As a result, they make the best decisions. In companies that compete on analytics, senior executives make it clear -- from the top down -- that analytics is central to strategy. Such organizations launch multiple initiatives involving complex data and statistical analysis, and quantitative activity is managed at the enterprise (not departmental) level."

Summary

One of the best ways to find web analysts with the three most important key elements is to attend a local Web Analytics Wednesday meeting. These are held every month in most major cities worldwide (including Montreal and Quebec city). The WAA also has a job board where companies can post their interest.

Another option, one followed by CSC Financial, is to grow your own. Companies following this route will want to make sure their candidate takes part in the WAA's UBC Award of Achievement in Web Analaytics trainings, reads the WAA message board and related blogs as they’re a ripe source of relevant information for people with experience and those just entering the field.

WaW, Omniture Café and something brewing!

Web Analytics Wednesday?

I received a lot of email asking about Web Analytics Wednesdays and if there would be one in June, before the summer vacation. I was hoping to organize it, but the workload and some logistics problems prevented me from planning it. So we'll take a break and get back to regular Web Analytics Wednesdays in September.

In the meantime, if you are in the Quebec city area, you might want to attend Yulbiz-Québec.

Something's brewing: Omniture regional training in Montreal

Following the success of the first Omniture Café in Montréal, I've been discussing with Omniture about regional training in Montréal.

If you had the opportunity of attending Omniture's training in Montreal at the end of September or early October, would you be interested?

The training cost would be roughly $1,500 for 3 days (exact pricing info might change).

Please fill-out this quick survey to help me with the planning.

Disclaimer: Although my employer (Desjardins General Insurance) is interested in this training, the planning is my personal initiative and my employer supports me in doing so. Survey answers will be shared with Omniture only if there is sufficient interest for this event. (there's an option at the end of the survey to keep the info confidential).

What should eBusiness Architect do?

Whenever I think of something interesting, I create a draft entry in my blog and get back to it later. I started writing this entry about a week ago, and this morning while swifting trough 100+ Google Reader entries, I noticed the post from Holly Buchanan: "If Architects Had to Work Like Web Designers".

There was also that interesting discussion thread on the Web Analytics group about "Does a web analyst have to know how an I.P. address is constructed?". Although useful, I don't think it's essential and you can read my opinion and those of others in the discussion thread.

A thing leading to another, I got an email that sparkled some interesting discussion about the role of an ebusiness architect. I also realized some of my co-workers don't know/understand what is my role as a "senior ebusiness architect".

So? What's an architect?

Let's borrow from Wikipedia: "an architect is a person who translates the user's needs into physical, built solution. An architect must thoroughly understand the building and operational codes under which his or her design must conform. That degree of knowledge is necessary so that he or she is not apt to omit any necessary requirements, or produce improper, conflicting, ambiguous, or confusing requirements. Architects must understand the various methods available to the builder for building the client's structure, so that he or she can negotiate with the client to produce a best possible compromise of the results desired within explicit cost and time boundaries."

As we read this definition, it's obvious it perfectly applies to Information Systems/Information Technology and more specifically, to ebusiness and Web initiatives.

Some caveats (and a legal disclaimer)

In some countries, like Canada, job titles such as "engineer" or "architect" are regulated professions. So officially, I'm not a "senior ebusiness architect", I'm a "senior ebusiness advisor". Just like a few years ago, I tried to explain that I was not a "software engineer" simply because I couldn't use that title.

An architect for every soup

There's often confusion around the job titles. Put any of those words in front of "architect" and you've got a new field of expertise and a new career: software, technical, organic, functional... ebusiness, enterprise...

What should an ebusiness architect do?

Simply put, my role is exactly that of an architect: take the business requirements and plan for the solution. A bit too simplistic, let's look more closely at that phrase:

• take: this implies a lot of listening, communicating and explaining the process that will lead to the solution
  
• business: have the right interlocutor and that he/she is empowered to take decision/action
  
• requirements: one of the biggest challenge is the fact that "requirements" are often (almost always!) expressed in terms of "solution". From the requirements, we must strive to understand the initial objective. To take the home analogy, the client knows he/she wants a "3-section with side-panels triple-glass wooden-frame 6 feet by 4 feet window" on that wall, but probably doesn't know how the wall will need to be reinforced to support the 2nd floor. The objective would be something like "The dining room as a view on our garden and we can see really beautiful sunsets. I want the largest window possible as long as it doesn't increase the overall cost of the house more than X$". It's the architect responsibility to read between the lines and translate those requirements into realistic objectives.
  
• plan: It's also the architect responsibility to work within the constraints of time, budget and quality. This implies a vast understanding of the subject and the collaboration of field experts to gather the elements of the solution and mutually challenge the solution to come up with something as efficient and as realistic as possible.
  
• solution: Solutions are mutually accepted compromises. That last point is critical: sometimes it's the role of the architect to go back to the client and explain why the requirement can't be met, or why another approach might be better or at least, satisfy a fair percentage of the goal without necessarily constraining future enhancements.
  

Key skills of an ebusiness architect

Just like the construction architect, the ebusiness architect doesn't actually "build" the solution, but he/she has to take responsibility for it's conception and overlook the work so the job is done according to plans. Strong knowledge of the various work expertise, supporting technologies and best practices, trends and evolution of the ebusiness field, strong analytical skills and be solution driven, as well as acute communication skills are all essential ingredients of a good architect.

The feet strongly grounded in IT, the brain on the business side

One of the most interesting challenge of the ebusiness architect is to "translate" the business lingua into more technical terms, and vice-versa. We often see this latent tension between "IT" and "business", "us" and "them". Sometimes, the ebusiness architect is merely a communication gear between two quite different worlds. Being strongly grounded in IT while at the same time being able to walk the talk of the business is probably the most important role of the ebusiness architect.

Study results: 3 key skills of web analysts

I'm really proud to announce the publication of "3 key skills of web analysts" on iMediaConnection.

The role of web analyst is critical to a successful internet presence and maybe even the success of entire organizations.

This article provides the results of my study of web analysts and answers two primary objectives:

1. Quantify how web analysts spend their time on various tasks, and more importantly, what type of cognitive process they engage in while doing those activities.
2. Help web analysts understand their own job characteristics and explain it to their managers and fellow workers

Conducting such a study was an interesting endeavor and a very good learning experience! Thanks to Avinash for sparkling the idea with his post about "How should web analysts spend their day". I also got great support from John Beck, co-author of "The Attention Economy" and visionary at The Attention Co. Thanks to Jim Sterne, Avinash Kaushik, Jacques Warren, Anil Batra and my boss Bernard Cinq-Mars who reviewed and commented an early draft of the article. I specially want to acknowledge the great help I got from my friend Joseph Carrabis, who coached me into writing this article and offered his great wisdom to help me out. And lastly, Brad Berens, editor in chief of iMediaConnection, who's giving me the chance to reach beyond my little blog!

Of course, a special thanks goes to the web analysts who spared a little bit of their attention and took the time to fill out the survey. I look forward to comments, suggestions and even critics!

Stay tuned! In the coming days I will publish additional details about various aspects of the study.

Is it ethical to track individual details through WA?

Another very interesting discussion thread on the Yahoo! Web Analytics discussion group. The question is:

Let's say the marketing department wanted to review the behavior of each and every user of an internal application in the banking domain. Would it be ethical to use Web Analytics to track this information?

Ethical concerns are very different when dealing with the general public or employees, especially if within the boundaries of internal corporate systems.

Web Analytics and system journalizing

Web analytics should not be used to track individuals, but rather to find actionable patterns that you can act upon. Just like in a statistics sampling, you don't have to use all the data, and you don't focus on an individual entry, but you can still come up with valid conclusions.

Journalizing (to record in a journal) goal is to track the actions of an individual person. This can be a perfectly valid scenario where "non repudiation" of an action is important (such as in financial transactions). This is usually done at the system level and is stored in secured databases, not using WA solutions. For internal systems, especially in financial institutions, it is best practice to monitor abnormal employee activities.

Is it ethical?

In the specific question above, the IT department because they are very likely to already have some kind of logging and monitoring in place... and probably it's something not widely publicized within the organization. Now, from a marketing perspective, I don't think it would be ethical to use that data in to track individual user actions. Would it be acceptable for marketing to read each employee email in the hope of understanding what everyone is doing? Obviously not. Is it acceptable to monitor email activities to filter or act on viruses, spam, adult oriented or confidential leaks received or sent by employees? Absolutely. It should be the same for the use of web sites and other systems: act on the exception, not the individual person.

For more information on this topic, read my previous post entitled "Protect you customers' privacy ethically, not legally".

Google: Is there a God?

New version of a classic joke

One of the most powerful organization in the world decided to ask the question to it's hi-tech computer. "Is there a God?". After feeding in all relevant information available they typed it in and waited. After a lot of computing the system went into an eerie silence for a few minutes and then started typing.

It's answer was "Insufficient data."

Not to be outdone, the 8,000 engineers in their infinite wisdom started gathering all the information from the Worlds libraries, archives, and Web sites. So much information was assembled that it was decided that one computer could not handle it all so a network of computers were linked for the operation. Again the question was asked and all the computers went into action. After a couple of minutes the answer was forthcoming.

"Your search for - God - did not match any documents."

Calculated a bit more and said:
"- Make sure all words are spelled correctly.
- Try different keywords.
- Try more general keywords.
- Try fewer keywords."

This time they were going to get an answer to an age old problem and nothing would stop them. After months of negotiations with the most powerful companies and worldwide governments they were able to gather enough data and link all the computers in the world together to produce the ultimate super-computer. Nothing would stop them now. Just to make sure they fed in all information even remotely connected to the subject, they included every piece of personal information they could find.

The CEO typed in the question:



The computer whirred into action checking all it's drives and then linking with all the other computers. After seconds of activity going from one computer to another everybody waited eagerly as it typed to the screen.

The moral of the story

This joke highlights the point that too much power concentration, even with the best intentions, can lead to unpredictable results. As Google market caps continues to grow (and is predicted to surpass Microsoft), attention as already shifted from one super-power to another. At the same time, worries of abuse are also shifting from Microsoft to Google. While people worried about Microsoft monopolistic positions as a software platform, Google's privacy concerns are much more troublesome. This step is just the logical chain of event that historically grew from IBM's hardware to Microsoft software, and now to Google data.

Totally excluding Google from our lives is similar to saying we won't watch TV or drive our cars... Except for the most hard core people, this kind of tactic is unrealistic. Our hope is Google will understand there is a fine (virtual) line they shouldn't cross. They are starting to play with it and if they don't watch, the "2006 person of the year" could very well start to look at other alternatives.

Aren't we all founders at work?

For the past few days, I've been reading the book "Founders at work" by Jessica Livingston. An interesting read for would-be entrepreneur or anyone curious about the early days of some of the most successful internet companies. This got me thinking that all of us, in our own way, might be founders at work.

Book review

There's an interesting path from the early stories of Steve Wozniak working on the Apple or up to recent success stories like Flickr. Each short story makes only a couple of pages of interview with the founder: initial idea, funding, good and bad turns and advices to would-be entrepreneurs. That makes up a good and easy summer read.

What I would have loved to see is a short summary of each story key takeaways and a final chapter that could have provided an expert view and some advices to would-be entrepreneurs. "The Art of the Start" from Guy Kawasaki would probably be more appropriate for that.

Idea, guts, luck and connections

Although each story is different, there are some common traits:

• Everybody has ideas, few have the guts to take the risk of turning them into something more tangible.
• Luck is often a factor, although I believe one can create favorable conditions to increase the chances of being lucky.
• The initial idea often derive into something else, more interesting and more viable.
  
• Having the right connections is also an element of success.
• Diploma and experience doesn't seem to correlate very strongly with successful entrepreneurs, although entrepreneurs get more successful with each startup experience.
  

Missed opportunities?

Trough the years, I had some ideas that, given the right timing, the right contacts and a little more guts, could have (maybe) turned out into something big:

• While working at Softimage as a webmaster, I thought of a web site that would feature commercials, short movies, game animations and movie special effects that were done with the products. People would get a synopsis, watch the video, and rate it or send critics. Animators and students were eager to show off their work, so they would have submitted their own videos. I shared the idea with the founder and it was dismissed as impossible... a few months later adCritic was born... That was in 1998.
• While working as a project lead for the redesign of a dozen web sites, we worked with an interactive agency that sent us links to an online prototype. Business owners (marketing & communication people) would print the page in black & white, handwrite annotations for corrections, and fax it back to the agency... with cut-off margins, unreadable comments and bad handwriting! I thought of an online collaboration tool that would allow someone to put virtual Post-It notes on a page and share them with other people. That was around 2000. Check out Fleck.com

A new opportunity?

Now I've got this crazy idea about WASP, the Web Analytics Solution profile, that would ease the implementation of web analytics solutions. The early version, more a proof of concept than anything else, was received with great interest and I got very interesting feedback.

Could there be some business opportunity behind it? Maybe, not sure, but it's a hell of a fun to do it!

What's your own missed opportunity?

Apple releases public beta of Safari 3.0 for OS X, XP and Vista

Apple announced today the availability of the Safari 3.0 browser in public beta for OS X, Windows XP and Windows Vista.

Why is it significant?

The release of the iPod, a few years back, has brought back a lot of attention (and dollars) to Apple. With the MacBook astonishing popularity (and gradually steeling away users from Windows to Mac OS with the Boot Camp feature) and the upcoming iPhone buzz, Apple is building on its quality and ease of use reputation that has always been part of their culture. Safari 3.0 promise to be twice as fast as Internet Explorer, and 1.6 times faster than Firefox.

My first impression: I don't know if it's 3.0 times, but it's blazing fast under Vista!

Considering those who switched from MSIE to Firefox already know the drill, I suspect we'll see a lot of people moving from Firefox, or at least trying out Safari.

The web analytics perspective

Here's some impacts of this new browser from a web analytics perspective:

• Web design: So far, I haven't found any site that didn't display correctly. That's good news!
• Site functionalities: there seems to be some problems with JavaScript handling, but I haven't investigated it. I started writing this post using Safari under Windows Vista but had to switch to Firefox because I couldn't insert the image or save a draft... It's still beta.
  
• Technographics: we'll start to see a new browser coming up in our reports, you might want to watch for the adoption rate and who will suffer from it.
• Cookie handling:
1. the default option is "Only from sites you navigate to: For example, not from advertisers on those sites".
2. When private browsing is turned on, cookies are deleted.
   

Those last two point are very important. While building WASP, I noticed too many sites and web analytics solutions are still implemented with 3rd party cookies. This situation will only get worst (and unacceptable) in the near future. The "private browsing" option will inevitably impact the calculation of unique visitors.

Have you tried Safari? How will it impact web analytics?

WASP detection pattern

Thanks to some very valuable feedback, I will make some major changes to the WASP detection model.

Constraints

WASP is far from perfect. One of the most common comment I got was that it doesn't detect frames.

The other thing is each tool needs to be added to the configuration file, knowing there is actually anywhere between 150 and over 200 solutions on the market, this becomes a huge maintenance issue. Whenever an existing solution is updated or a new one comes to market, the configuration file needs to be reviewed.

The actual detection model relies on the presence of a specific JavaScript object (a variable or a function) and/or an URL pattern. This detection process is done after the page is fully loaded.

Proxy approach

I have found some patterns that are common to all tools, so I will change the way WASP does it's detection to work more like a proxy and detect the HTTP Get and the HTTP Response patterns. This new method should increase the confidence level and catch any tool that uses tags. Of course I will add more features for tools that are more commons (like Omniture, HBX, Google Analytics, etc.) but the nice thing is WASP will be able to detect even the most obscure tracking systems.

Stick to the objective

When I decided to build WASP, my goals were twofold:

1. Build an end user tool that would help regular surfers understand how their behavior is being tracked. This free version of the extension has two sub goals:
1. Reassure users that there is nothing evil with most web analytics solutions
2. Help identify sites that are abusing the user privacy or tools that are misconfigured
   
2. Build a more advanced tool that will help WA specialists do quality assurance and ease their implementation. This version (actually based on voluntary donation - but that could change), would include advanced features that not even the web analytics provider themselves are actually offering (I'm keeping that part voluntarily obscure...).

It's very tempting to widen the role of such a tool, and some of the feedback I got was very interesting. But while reading the book "Founders at work" (stay tuned for my review of this book by Jessica Livingston) I noticed a lot of successful projects started from an initial idea and evolved to something even better. Once what appeared to be the "yellow brick road" was identified, all energy went to it.

Google AdWords Conversion hick-ups

It's common practice to add the JavaScript code snippet to the conversion page of a campaign that was initiated trough AdWords. This way, we can track the whole conversion process from its PPC origin all the way to the conversion. Fine with me.

However, recently (June 1st?), Google changed its policy and now a subtle but intrusive and annoying little box shows up on the conversion page.

Google AdWords invasion

Here's Google explanation about "Why is the conversion tracking text visible?":

We want to notify the user that his/her site activity is being tracked. However, because we also want to protect the user's identity, we also make it clear that we don't record or use the user's personal data in any way.

And "Why use visible text for conversion tracking when everyone else uses an invisible image?":

Google has chosen to use visible Google Site Stats text for the conversion tracking process to make this process apparent to users. Our goal is not to place our brand or promote our company on your web pages; however, we feel that it is important to be honest with and respect the privacy of users being tracked.

For example, if users see and click on this text, Google provides them with information about not accepting the conversion tracking cookie if they wish, and we assure them that none of their personal information is being used or recorded in any way. Users can also learn more about conversion tracking itself by clicking on this text, and will also have the opportunity to provide feedback about their experience with your site.

We made the Google Site Stats text block as unobtrusive as possible, and ask that you place it in a corner of your confirmation page (the page after a transaction has taken place).

Isn't it the responsibility of the "host" to have the proper privacy policies and disclaimers? In the case of AdWords conversion tracking, Google is merely an utility library being included in "our" site, it is not "their" property. They don't have any "ownership" of the web page. If we're using other "tools" to build our web site, are we going to end up with a bunch of little boxes all claiming their own part of the page and the user's attention for the sake of "protecting privacy" and "making it clear"? This looks unacceptable to me.

Broken page rendering

This doesn't seems to be a problem on our end since I have noticed other sites with similar issues. Because of this, I have already noticed some sites with screwed page rendering because what was a non-intrusive piece of code (without any display) has now become this annoying little box that shows up wherever the code was included.

Statistics & feedback

The "Google Site Stats - send feedback" box (or simple text link) leads to a simple explanation page explaining how the user is being tracked.

What do you think? Anyone else noticed that? Do you think it's fine or are we using Google AdWords Conversion in the wrong way?

WASP 0.23 feedback

It's been a week since I released version 0.23 of WASP and the response is amazing!

Stats

This is all about web analytics, some here's some stats:

I have doubled the traffic to my blog, but more importantly, my conversion KPI is amazing: WASP has been downloaded over 7000 times in a week!

Rants

What would be any piece of software without some rants?

I have just installed the updated version and I like it a lot. It is kind of fun to see what solutions various websites have and even find some interesting ones (the BBC site is using Nielsen/NetRatings!).

Thanks for all the effort you put into this on behalf of our industry.
Avinash Kaushik

Compared to a batch oriented analysis of FiddlerTool.com captures that I have scripts for, it might be a lot easier to use and so would be something I’d recommend to people debugging analytics problems.

George

WASP is just plain COOL! As a professional web analyst, I use it all the time. It is very useful; I can’t imagine an implementation without it.
Jacques Warren

Feedback

If you had a look at WASP, I would like to hear from you. What do you find useful? What would you like to see improved? What did or didn't work?