Web metrics can mislead you and how to avoid it

Web metrics can mislead analysts including the Wall Street analysts.  At one point in time, Wall Street ignored comScore’s disclaimer on the limitations of its metrics data, causing people to expect lower Google’s revenue. It turned out that Google had stronger earnings for the first quarter of 2008. Here is the article from NY Times “Web Metrics and Grains of Salt”. Web metrics data could be much helpful if more questions were asked and challenged.

The key takeaway from this incident, for me, is, never settle for one main data point and question beyond the given metrics put in front of you.

In this incident, it would be comScore reports showing a slowdown in paid clicks on Google’s sites. If you’re a web analyst, it is important to ask yourself beyond what is given.

I highly doubt those Wall Street analysts only gauged on that one specific data, but when it comes down to analyzing web metrics, there would be far more questions arising out of these data. (I’ll limit it to few key questions since the list can go on forever)

• In this specific scenario, is comScore limiting “paid clicks” to just Google’s SERPs (search engine result pages)? Google’s content targeting ads are placed on so many other websites, and that should automatically tell us that Google’s revenue isn’t just limited to their domain.
• comScore’s projection was based on United States market, how much of that would be accounted in Google’s revenue?
• Do increase/decrease in clicks automatically associate to higher or lower revenue? I don’t think so if Google says the quality of clicks will be enhanced, there is a possibility that each click could weigh more in terms of money. In another word, enhancing for qualified click-throughs mean higher conversion, and potentially allowing Google to generate more money (in areas of referrals, high-value keywords, competitive keywords, etc.)
• Traffic from other countries besides the United States probably offset slower growth, but if you’re not sure what Google’s revenue drivers are then, how can you assure lower revenue from such company?

Although I’m talking about Google in general in this case, the kind of thinking process to ask beyond a data point to generate next questions is an important process in digital analytics.

So this article by NY Times, states at the beginning “Perhaps Internet metrics should come with a warning label: “handle with care.” is actually not a bad idea at all.

Digital marketing analytics are great to analyze “what” is going on with the site, but it always has been a challenge to compare it across different software platforms, different medium, and even answer the “why” customers behave in a certain way.

So when using web metrics to associate data across different platforms or environment, it would probably be wise to count on analysts with the digital background than analysts who has no clue on how digital marketing technology work because web metrics data needs to be handled with care.