Perspective on web site engagement effectiveness

There are multiple ways to measure the effectiveness of a web site, but few ways to get a holistic perspective but to rely on multiple metrics sources and analysis techniques.

User Experience metrics

One obvious and often used method is to poll the users about their satisfaction level: straight and effective. This qualitative data, however, is often highly suggestive and not standardized.

From their own perspective, web analytics solutions provides valuable quantitative information about the behavior of the users on the site. We get a broader view, from the user origin up to his or her exit point on the site. The challenge, for many companies, is to correlate this information with the off line activities taking place either before or after the user's visit to the site.

Web site engagement brings a newer perspective focused on the site visitor. Peterson, Marshall Sponder and Gary Angel are some of the leaders discussing this analysis method, which encompass the following metrics:

1. Click-Depth Index: Percent of visitor sessions of "n" or more pages
2. Recency Index: Percent of visitor sessions occurring in the last "small n" weeks
3. Duration Index: Percent of visitor sessions of "n" or more minutes
4. Brand Index: Percent of visitor sessions originating directly or originating from search engine searches for a significant term or phrase.
5. Blog Index: Ratio of blog reading sessions to all sessions
6. Conversion Index: In this case, session- or order-based conversion.

Infrastructure metrics

Information about the health of the infrastructure provides another angle to help us analyze causation and improve the site. Sadly, lots of organizations still live in a world of great divide between their "business people" and their "IT techies" department. Perceptions and attitude often creates a cleavage between marketing, communication or sales departments and their IT department. On the Web, like in many other situations, the business objectives can only be realized with the support of technology...

Strategic objectives

The last angle, maybe the most important, comes from the business side analysis, the real bottom line: is the site contributing to the core business objectives? Too often, we get generic objectives for the Web site: "I want to increase sales" or "I want to reduce costs"... who doesn't? We need to first understand the business side of the equation, without ever saying a word about the Web. Then, and only then, should we see how (and if!) the web can contribute to those objectives.

Holistic perspective

It's only when you can correlate and explain the relationships between the user experience, the infrastructure, and the business that you really start to get a holistic view of web analytics. That's essentially where "reporting" grow to become "analysis". Reporting is factual and often help to explain anecdotal episodes, while analysis is much more valuable in it's approach from root causation up to strategic recommendation.

Where does Attention fits in?

The previous perspectives covered the user, the technology, and the business. But where do I bring the Attention Economy concepts?

The user is not merely a "unique visitor", he or she is a human being with some needs to satisfy, some goals to accomplish and living emotions. In the concept of Attention Economy, the scarcest resource, and the most valuable element, what we're seeking, is our fair share of Attention. The problem (or interesting thing!) is attention takes different forms:

• captive or voluntary
• front or back of mind
• aversive or attractive

The example below, taken from The Attention Company study on attention types for various media channels, clearly shows the user behavior is different. Knowing this, each media is better positioned to address those differences and try to transform an aversive experience into a more attractive one. Or a captive activity into a more entertaining and voluntary one.


In future posts I will describe how this concept could be applied to better understand a specific web site.

Marketing analysis with RFM

This is the second post in a series on the concept of Attention Economy.

Marketing analysis using RFM (Recency, Frequency, Monetary value) has been used for over 35 years to target the segment of customers most susceptible of answering a marketing stimuli. It is often considered for emarketing segmentation, mostly for email campaigns.

One of the interesting aspect of RFM comes from it's simplicity and ease of application. From historical purchase data, each customer is classified in quintiles (20% slices) by looking at the last purchase date (Recency), the number of purchases (Frequency), and the overall purchase value (Monetary). The Pareto principle applies in most cases: we often find out that 80% of the purchases comes from the 20% of the customers.

The "cube" picture to the right, borrowed from Jim Sterne's classic paper on e-metrics, gives a clear representation of the usefulness of the RFM classification.

However, RFM also has it's limitations. Although past behavior might be an indication of future attitude, "tripwires" should be set carefully to avoid frustrating the user. There's also a risk of over soliciting the same cream of the crop over and over again.

A potential customers typically "converts" when a purchase is completed. However, in the web world and depending on the web site objectives, the "conversion" could be any significant business event. On a ad-based web site, the monetary value could become the click-trough rate and the ad-revenue generated. A support web site could evaluate the value of "cost avoidance" of self-service vs. call-center based interactions, etc.

The next post will provide a perspective on web site engagement effectiveness.

RFMA - Recency, Frequency, Monetary & Attention

In just a couple of years, Web analysis concepts have evolved from counting server hits to page views, visits and visitors. Sophisticated web analytics solutions such as Omniture SiteCatalyst, WebSideStory HBX, Coremetrics and others can establish correlations between user behavior and site characteristics to visually present them in powerful, yet simple to understand reports and dashboards. Yet, we've seen a growing concern that the "page view" might not be enough to measure the sophisticated and various ways information can be disseminated and used without the constraints of time, space and medium. The death of the page view is a hot topic in web analytics, and expert groups are looking into other ways to measure success in the era of Web 2.0, RSS, blogs, mashups, YouTube and unexpected or even undiscovered new ways of using information.

At a macroscopic level, numerous methodologies have been conceived and tested to measure the success (and too often the failure!) of ebusiness and emarketing initiatives. Brought back at an individual-level, that is, at the level of the human being interacting with information trough a media/medium; there is little objective ways of measuring the quality of the interaction. Measuring the value of a relationship has become a necessity. The challenge isn't merely to build a customer-value model, but to use it in a sophisticated and evolving analytical environment.

What if we could use a simple marketing classification method such as RFM (Recency/Frequency/Monetary value) that has been used for over 35 years, and add a degree of Attention to drive the relationship value model? In this context, "Attention" can be defined in several ways, but let's simply consider that "attention is a psychological construct describing detection, selection, discrimination of stimuli, as well as allocating of limited processing resources to competing attentional demands".

In the coming weeks, I will explore some analytical concepts, web or otherwise, I first studied four years ago as part of the MBA course "Understanding the digital enterprise", presented by Professor Stéphane Gauvin, from Laval University. I will also demonstrate how the RFM model can be used in web analytics, and add the concept of Attention as presented by Goldhaber and others. But the bulk of the model will be validated with the analysis method elaborated by Davenport & Beck in the book "The Attention Economy".

I can't pretend to be an expert in the field, maybe just a bit more fanatic about web analytics than most people. Considering some very bright people are already at work trying to find ways to measure such things as "engagement", my views might sound too simplistic or theoric at times. I will nevertheless dig the topic for the pure interest of it. In this journey, I will encourage you to share you thoughts and ideas, even if it's to tell me I'm crazy :)

A case about reporting vs. analysis

When designing a web site, we need to understand the demographics, psychographics and other characteristics of our audience. Elements to consider includes such technical considerations as the bandwidth, browser, and computer being used.

In this example, I will use Omniture SiteCatalyst but the same type of analysis could be done with any solution. For the sake of this demonstration, let's assume it's a B2B site, so visitors comes from very specific partner companies.

I was faced with a very good case where analysis, instead of reporting, made all the difference. Take a simple statement: 15% of the visitors to the web site use Windows 2000/IE 5. We know this could lead to slightly different design considerations.

If we are simply reporting, we're stuck with 15%... is it worth designing for this platform? How can we tell?

Since we suspect Windows 2000/IE 5 use might be declining, let's look at the usage trend for our visitors. We already have a clearer indication, going from nearly 16% down to 12% in 5 months. Plus the fact Omniture can tell us the Internet average is about 6%, indicating our site receives a larger than average share of Windows 2000 visitors.

Could there be just a handful of visitors (companies) using Windows 2000 and influencing this reporting? Now we need to create a Custom Datawarehouse Report using Omniture's Advanced Analysis. We can ask for the number of visits by OS, grouped by Company, with a breakdown by month. Once we get the CSV file loaded in Excel, a simple Pivot Table can give us a very good look at what is going on.


Now we can spot that only a couple of companies, some of them representing a fair share of the traffic, have much higher usage of Windows 2000 (42% in the above example). This makes sense because it appear one of those companies is a banking environment, and we know those corporate environment traditionally takes longer to adopt more recent technologies. Furthermore, we see there's also a clear negative trend, from 122 visits in October to 54 in February (while XP is generally growing).

Now that we have done a more thorough analysis, we're in a much better position to decide if we need to invest in designing for this platform. From a simple report stating "15%" with very limited context, we were able to identify more detailed behavior and address the impact on a limited subset of our visitors.

In the spirit of the 10/90 rule coined by Avinash Kaushik, getting the number was easy. The analysis is where the value is: how is the site structured? who's your audience? which other reports and metrics will help you get the right decision? Too often, people are complaining about the tool: it's not working, it's not configured correctly, it doesn't give me my numbers... I guess that's where experience and an analytical mindset can make all the difference.

(Ultimately, the recommendation was to avoid integrating costly exceptions in the design and the development process.)

March Montreal WaW Recap

Yesterday's Web Analytics Wednesday brought about a dozen people, a bit less than the expected 20 or so, but nevertheless some new faces and enough to have very good discussions. Here's a brief recap of the meeting, but if you want a very detailed one, check out YashLabs detailed account.

After the usual chit chat and a round table where we presented ourselves, I was happy to share a recent talk I had with M. Scott Wells, senior account executive from Omniture, where we discussed the possibility of having an Omniture Café event in Montréal. I think this is exactly in the spirit of the Web Analytics Association objectives: unite web analytics consultants, professionals and end users to increase our common knowledge and best practices in web analytics.

To give a new twist to our get together, I invited the participants to pick a topic from the WAA course on web analytics for site optimization I will present at the coming eMetrics conference. Following a brief intro, everyone was invited to share their thoughts and ideas about "designing for usability or conversion (sales)". Although we agreed one is obviously not incompatible with the other, there are some distinctions between the two mindsets when comes the time to build a web site. Being usable doesn't necessarily mean it's built for your audience. Brian Eisenberg example is self explanatory:

Usability is related to the individual's subjective experience. The same mug I love to drink from is a disaster waiting to happen for my two-year-old daughter. Is the mug usable? Yes, for me. Not for her.

Later on I had a good chat with Sean Power, a fellow Web Analytics Association volunteer, about our involvement and the positions we're in: him at Coradiant, and myself as a part-time practitioner at Desjardins. When I left my previous employer I was very close to start on my own as a web analytics consultant. For now and for the foreseeable future, I'm happy with the choice I made: a good job and an outstanding company culture, good work-family balance, and support for my activities in the web analytics community. What could I ask more? :)

P.S. You can get your own WAA mug and other stuff from the new CafePress store!

Usability vs pseudo security

Today I was planning my trip to eMetrics and I learned that my employer asks us to be Aeroplan members. My point isn't about the benefits of Aeroplan, but rather about a usability problem that seems to stem from a false sentiment of security/antispam policy: the captcha concept.

In general, my opinion is there is too much lost space, which puts about a third of the content below the fold. But there are some small nuggets of mis-usability in there too! I'm sure you can find some of them on the enrollment page: check where it says "call us"... what's missing from the page? A phone number! It's nowhere to be seen... Can you find others?

But the worst comes when on the last step, it asks to type what is shown in the image. Their justification is reasonable:

Another security measure, Image Validation allows Aeroplan to be certain that a real person, you, is creating the new account, not a program seeking to achieve false enrollments. Programs and computers cannot read the characters in the validation image, so cannot pass this test.

There is just one big problem with this approach:

Can you read the picture shown below?

I tried... but I obviously made a mistake because it came back with this one:

Not much better...

I think they should read the book Don't Make Me Think.

Would you recommend using captcha?

Web Analytics Wednesday, March 14th, Montreal

March 14th Web Analytics Wednesday is coming up!
English version down below.

Comme le veut la tradition du Web Analytics Wednesday, ces rencontres sont de natures "informelle" et il n'y a généralement pas d'agenda précis. Les discussions s'amorcent librement et vont aux grés des intérêts de chacun.

Le tout ce veut chaleureux, simple et une occasion idéale de rencontrer des gens qui partagent des intérêts communs autour de l'analytique Web, de la conception de sites, du design, de l'ergonomie, du SEO, et même des statistiques en général, des aspects éthiques et légaux, etc.

Ça vous intéresse?

Lieu:
Le Commensal
1720, rue Saint-Denis
Montréal, QC H2X 3K6
514-845-2627

Nous aurons une salle privée à partir de 18:00h.

RSVP: réservez en me faisant parvenir un courriel, en laissant un commentaire ou en ligne.

Note: Si vous aimeriez présenter un service ou un produit, être un "sponsor" en offrant une tournée ou un cadeau de présence, ou simplement faire une petite présentation sur le sujet de l'analytique, SVP me contacter. La seule restriction est que la présentation doit viser l'éducation et la croissance professionnelle des gens, et non de promouvoir simplement votre produit ou service.

---

The monthly meeting of Quebec's Web Analytics community is inspired by the Web Analytics Wednesday tradition, which is an happy gathering of practitioners in the field of web analytics or other related fields of interest (SEO, design, or the Internet in general).

For the month of March, we will be back in Montreal, on Wednesday, March 14th.

Location:
Le Commensal
1720, rue Saint-Denis
Montréal, QC H2X 3K6
514-845-2627

RSVP: simply send me an email, leave a comment below or register on-line.

Sponsors: If you would like to sponsor this event, present a product or a service, pay for the first round, or simply present something on the subject of web analytics, please contact me. The only restriction, to stay in the true spirit of WaW, is that your presentation be educative or otherwise help people grow in their web analytics practice, not simply to promote your product or service.

Cool stuff for February

Last month, I posted about cool stuff for January. As I surf, I discover new services, sometimes nuggets of interest on someone else's blog, sometimes stumbling upon something while doing research.

AniBoom

My 15 year old daughter is really into visuals arts and she does fantastics portraits. She also likes animation and she's thinking of pursuing her education in this field. We found AniBOOM to be a fantastic place to discover very cool animations done by professionals and students. Think of it like something akin to YouTube for animated short stories. Some pieces worth looking:

• The way of hope
• End of summer
• Dragon slayer
• and dozens more!
  

Everything Web 2.0

I sometimes feel like we're back in the pre-bubble era. Check out how an exciting era we're living by scooting over 900 Web 2.0 applications classified and tagged for your greatest pleasure and hours of discovery at Go2Web20.net. And if you want to see who's cool and who's loosing ground, head to Movers 2.0.

Data visualization

You might have heard about Swivel, a place where you can explore, share, compare and upload your own data sets and use visualization tools. A similar tool, Many Eyes offers some amazing visualization tools that are more dynamic than Swivel. If you are into data visualization, both are worth a look.

Web geek tool

Web analytics should always be correlated with information about the health of the systems supporting the site itself. Internal monitoring and reporting is important, but sometimes you want to get a better feel of the user experience: just how painfully slow the site can be. Octagate SiteTimer offers a free tool that comes very handy to get a detailed breakdown of each components of a Web page and the time spent waiting and downloading. Other tools offer this kind of breakdown but are part of larger service packages (Gomez, Keynote, etc.). SiteTimer works, it's simple, and it's free.

RSS Feeds + news + social network = myFeedz

I discovered myFeedz, from Adobe Labs, just a few days ago, but I think I will like it. It merges concepts from Google Reader, Feedburner and Technorati into what promise to be a very powerful tool. Ratings, tags, bookmarking are all there, and unlike other feed readers, myFeeds tries to learn from what you read regularly and accordingly displays news items / blog posts that might be of interest to you.

What's keeping me busy?

If you have noticed a slowdown in my blog posts, it's not that I'm not interested in web analytics anymore, to the contrary!

eMetrics is coming up!

I will be presenting the workshop on web analytics for site optimization. Working on the 3 hour session is a daunting task in itself. There are so many things I would like to talk about! Check out eMetrics in San-Francisco, and if you are from Quebec and plan on going, let me know if you go.

Web Analytics Wednesday still going on!

The next WaW is scheduled for March 14th, in Montreal. I have to confirm the place, send the invites, and see if anyone is interested to present something. If you plan to attend, have suggestions for a good and quiet restaurant, or would like to present, send me a note.

Web Analytics an hour a day

I won't repeat what fellow bloggers have said about Avinash's upcoming book (here, here and here). Just get it!

WASP

I'm putting the development of WASP on hold for the obvious reasons above.

Higher education

My MBA is still going on, although I've been neglecting it a bit. I will have lots of catch up to do next week!

...and the job!

As a senior web architect at Desjardins General Insurance, I'm working on a full redesign of our site (full as in "new objectives, new technology, new design, new content). This includes the deployment of MS Office Sharepoint Server 2007 and related technologies. So this is what keeps me busy at work, it's a really cool project.

WASP v0.21 on addons.mozzila.org

It took a while (over a month), but WASP is finally available directly from the Firefox addons site. The delay was caused by Mozilla working on a major update of the site, which increased the queue to over 300 addons to approve.

In the future, WASP will always be available from there. This should not only increase the visibility and usage of WASP, but also make it easier to upgrade when future releases become available.

I'm still working on the next update, which will be able to analyze more solutions, provide new features and tweaks to the detection engine. I'm also keeping a couple of nice features secret because I don't want to commit to them right now, and to be honest, I'm not sure it's going to work! But if it does, it's going to be really cool!