We’ve written in the past about how to measure ROI in online communities. It’s a subject we return to often with our clients at FreshNetworks. The online communities that we build for them all tie into over-riding business aims, and so measuring the impact is important. We can, of course, measure specific insights that they get from the community, the benefit of the qualitative information internally, the benefit that support communities have or any uplift in sales from the community. But there is a holy grail in online communities and indeed across social media - measuring ROI at a granular level; identifying influential members, recognising that these may not be those who post most.
In previous posts, I’ve suggested that what we need to do is develop a weighting that could be applied to individual members showing how important and influential they are. An analysis of the quality (not quantity) of their connections and of their connections’ own connections. A difficult and time-consuming task. And one that Google may have solved.
The latest edition of Business Week reports that Google has a patent pending on technology that measures influence in social networks. It apparently measures both the direct influence people have in terms of volume of connections, but also how successful your posts and feeds depending on how many people open, read and forward them.
The new technology could track not just how many friends you have on Facebook but how many friends your friends have. Well-connected chums make you particularly influential. The tracking system also would follow how frequently people post things on each other’s sites. It could even rate how successful somebody is in getting friends to read a news story or watch a video clip, according to people familiar with the patent filing.
It will be intriguing to see how this technology develops and what Google use it for. The measurement of influence online is of critical importance to brands, marketers and advertisers alike. Brands want to know how influential people who talk about their brand are, or how influential the people in their online community are. Marketers want to find these influential people and focus on what they are saying and what brands are saying to them. Advertisers can use this information to help target ads across social networks.
Of course, there must also be a benefit for Google. Given that their attempts at running their own social networks have not had the same success in sheer numbers as the likes of Facebook, MySpace and Hi5, Google is looking for other opportunities to capitalise upon this growing trend. They’re doing what they’ve done to the web - they don’t provide all the content they just offer a great way to search and prioritise it. So Google could become the Google of social media.
The latest edition of the McKinsey Quarterly includes the results from their global survey of Web 2.0 in firms. The survey documents the developments that we see at FreshNetworks - more firms are using more Web 2.0 tools for more complex business purposes. McKinsey go even further, noting that a significant finding from this year’s survey is that:
Companies that are deriving business value from these tools are now shifting from using them experimentally to adopting them as part of a broader business practice.
Web 2.0 tools are starting to enter the mainstream in business, those who trial them find them beneficial and want to look at ways they can use these tools across their business, helping them meet multiple aims.
If you’re interested in some of the detail of the McKinsey study, I’d suggest you go to the article on their site here (you will need to sign-up, although it is free). However, for me the most interesting findings are:
More community-based tools are growing in their use. In 2008, 34% of businesses studies used blogs (compared with 21% in 2007); 32% used wikis (compared with 24% in 2007)
Web 2.0 tools are popular both for internal purposes (94% of firms studied) and for interfacing with customers (87% of firms studied). When they are being used for the latter purpose, this is primarily to improve service to existing customers and then as an acquisition tool
Blogs were more popular in Asia-Pacific and India; social networking particularly popular in North America and China; and, mash-ups and rating more popular in Europe
The biggest barriers to using Web 2.0 tools are a lack of understanding of the financial benefits (28% of respondents), internal cultural barriers (22%) and lack of skills (17%)
This last point, the barriers to adoption, show the areas where we as an industry need to focus our efforts to help clients. We have written before on this blog about measurement and ROI in online communities and in social media (see posts here, here and here) and it seems that this is the biggest barrier that firms need support with. Perhaps as these firms move from trialling the use of new tools, to using them for specific business purposes, the measurement of how they contribute to these will be easier.
I’m returning to the subject of measuring the ROI of online communities after an all day session with a client today. We spoke a lot about how they are to measure the benefit of the online community they are launching and about the metrics they are going to measure and the KPIs they will report on.
I’ve worked with a lot of clients and read a lot of things on metrics and KPIs and have noticed that many people confuse these two terms. Really they’re very different. The best description I have seen of how and why they are different is a presentation from Dennis Mortensen, that I’ve copied below. It’s worth a look as it quickly and clearly shows how and why metrics are different from KPIs.
This ties in to the debate on measurement in online communities. There are many things that you can measure (metrics) but only some things that show the impact your community is having vis-à-vis the business or other objectives of the site. These are KPIs and are what you probably will be using to show the success of your online community.
So work out all the metrics that you can track with your online community, and work out what KPIs you are going to report on to show the success of your community. Both are measurements but only the latter is really a measurement of success of ROI.
The answer to this question is simple and complex. The simple answer is that you make sure you establish a set of criteria when you launch the community and then judge success against these. The complex answer come with some of the ways in which you may judge success.
The workshops that I’ve run with clients at FreshNetworks only prove that every online community has a different set of aims and so needs to be measured in different ways. However, a few basic principals exist that show the possible ways in which you can measure ROI.
First of all it’s useful to distinguish between actual measures of ROI and measures of the health of the community. As people who build and manage online communities, at FreshNetworks we are interested in metrics such as number of people who log-on each day, proportion who make contributions, average amount of time spent on the community and other such measures. These help us to benchmark the community against others that we have run and understand how each community is developing. It’s also useful for us to be able to build a set of metrics across community types so that we can better understand consumer behaviour in them. In most cases however, such measures are not (just) what we use to measure ROI.
A client may want to own the debate in a certain area. It may want to work with its customers to improve or develop its product in an online research community. It may be looking to engage a new audience demographic. It may want to amplify the word of mouth for its services. There are many reasons why people may set up online communities and understanding these on a very granular level is the first stage to effective measurement of ROI.
Then it’s just a matter of work out which factors will help you to understand how you are performing against these aims. There are really two types of data you can measure in a community
Qualitative data - this could mean measuring the quality of conversations about the brand, the response to the brand and competitors, the increase in actual conversations and dialogue about the brand, the extent to which these conversations are positive, the quality of ideas generated. Whatever you’re trying to achieve there will be qualitative measures that help you to measure this. Of course the problem with qualitative data is exactly that - it’s qualitative. Although this is the nature of online communities - they’re a space for conversations and discussions. So we often find that establishing detailed qualitative measures of success can be the most revealing and the most informative.
Quantitative data - to some extent this is much easier to measure, the trick is measuring the right thing and not measuring too much. From Google Analytics to more bespoke packages there are ways of quantifying a member’s interactions with the site and building a set of measures from this. It may be about reaching more people (possibly of a certain type), or about engaging the same people for longer. Quantitative measures can be good for understanding how people interact with the site.
This is really the fundamental difference between these two types of data and these two ways of measuring. Qualitative data helps us to understand the quality and usefulness of the contributions to the online community. Quantitative data, on the other hand, shows us how people have interacted with the site.
Most are important to measure and different clients have different aims from their online community - this means that different baskets of measures are needed. Sometimes you’re more interested in the quality of debate and so want to include a greater number of qualitative measures; sometimes you’re more interested in the reach and penetration of the community and so want to include more quantitative measures.
Whatever you measure it is that first question that’s important: what are you trying to acheive with this online community; how will it help your business? Answering that is where it all begins and one of the most critical stages in planning a successful online community strategy.
Next week I’m speaking at the SocialMediaInfluence conference in London on Measuring Influence and Audience online. It’s a tricky subject and looking around today I have been unable to find any examples of an approach which has been successfully and repeatedly applied.
The problem appears to be that whilst there are a whole range of metrics that we can measure in social media (see The Social Organisation blog for a fairly comprehensive list) but none of these truly gets to the crux of the problem. What we want to do is know is to measure the influence that a single blogger, commenter or video upload has. What is the value of a blog post praising Coca-Cola in terms that Coca-Cola could understand and measure. As many of our clients ask us, what’s the ROI of encouraging this kind of activity.
The answer is that it’s difficult to measure, not because we don’t have a range of metrics (we do) but because at the moment our understanding of what causes a particular post or a particular individual to be influential is limited. We can measure proxies, such as trackbacks, links to the site from other sites (and the number of links to the sites that link there). But these really only reflect an inherent influence that we still haven’t measured.
What we really want to know is how influential is everybody that is exposed to an piece of content, and how influential are all the people they influence. Of course calculating this number would be difficult if not impossible. And the information you need to gather would be huge. It really wouldn’t be worthwhile.
Which is why some more basic measure is needed. Take the sites like Dell’s Ideastorm and MyStarbucksIdea. These get peers to vote posts up or down depending how relevant they think they are. You can then migrate only the more popular posts to the front page or the top of the list. This kind of rough approach might be a crowd-sourced way of measuring influence. We know that the most popular posts are those that people in the community think the brand needs to listen to most. Perhaps this is the only measure of influence we need.
People talk about your brand everywhere online. A search in Google would find hundreds if not thousands of places where people are talking about or commenting on your brand. You could be mistaken for thinking that the rise of social media has led to people being exposed to more discussions about your brand that you don’t control. And you might think this is dangerous and get concerned about each of those hundreds or thousands of discussions.
But the mistake in this analysis is to assume that every discussion is equal. They’re not. A blog that criticises your brand but is read by hardly anybody is of very little importance to you. In fact the majority of discussions about your brand will each only have a very small audience. You don’t need to occupy yourself with them all, but rather with those which are repeatedly discussing you, or those that have significant influence.
This raises the question of how you measure influence. Aside from a very small number of well read bloggers, it is difficult at first glance to identify how influential somebody is online. You could look at how many posts they’ve made on a forum - but if nobody is reading them then that’s not a great measure.
Of more use is to measure the network effect that an individual has. How many forums or blogs are they active on and how many people read their posts on these sites? How many social networks are they a member of and how many friends do they have there?
These measures are much more useful when it comes to measuring influence - if somebody posts something about your brand how many different people will see it and how much will they trust the poster?
To some extent these measures can be made into a simple formula. Take each social network that somebody is a member of and give it a weighting for importance - so one that is more specific to your brand or product might be weighted more than a generic social network. So, taking a brand that sells baby products, a social network specifically for mums might be weighted twice as heavily as a Facebook. You can then multiply this number by the number of contacts you have on each site to get a measure of influence by site. And sum them to get a measure for the poster overall.
For example, Poster A has 100 friends on a Mum’s site (weighted 2) and 350 friends on Facebook (weighted 1) would have a total influence measure of 550. Compare this with Poster B, with 150 friends on a Mum’s site but only 70 friends on Facebook. Their influence measure would be 370.
So Poster A is about half as influential again as Poster B.
This is a good basis for a measure of influence and a way of finding the people you need to track and engage online. The problem at the moment is that the data needed to make this formula work isn’t readily available. Back to the drawing board maybe.
Mark Beth Kemp from Forrester thinks that the lifetime value of a customer is no longer enough. Both this and the ROI model are deficient because they don’t take account of customer dynamics.
One interesting example she gave was about the launch of the Wii. They wanted to launch a games console that appealed to an expanded market and to sell the benefits of gaming. One critical barrier they saw was the difficulty of persuading parents of children (specifically mums) that this was a new games console that kids should enjoy using. They turned this barrier into an opportunity and launched the alpha mums programme. Finding the most active mums online and seeding them with the product. These mums were encouraged to host parties where other mums could try the Wii. The result was incredible and shows the power of harnessing customer dynamics. One alpha mum alone sold 200 consoles to her email contacts.
For Mary Beth, your new high value customers are not those who spend the most with you over their lifetime. It’s more complex. The highest value comes from your Ambassadors - those who spend a relatively large amount personally and also have a lot of social value - they have a lot of contacts and have influence over these people. Finding exactly who these people are can be complex although really it just relies on finding out where they socialise (online or offline) and how many people they are in contact with there. This gives you their reach. You can then calculate how valuable their recommendations are likely to be and that gives you a crude measure of social value. Perhaps a developing but more useful measure for marketeers.
This morning we spent some time with Robert East, Professor of Consumer Behaviour at Kingston University. His research focuses on word of mouth and brand loyalty and we spent the morning talking about how you can measure these things and in particular critiqued the Net Promoter Score as an approach to this.
One of the outcomes from the session was an initial set of five principals to bear in mind when trying to measure word of mouth. We’re going to be bouncing these around and building on them internally at FreshNetworks, but if you have any comments or can add to the debate let me know:
Don’t confuse word of mouth with satisfaction - only a small proportion of word of mouth is prompted by a satisfactory or dissatisfactory experience
Measuring word of mouth by the likelihood to talk (positively or negatively) about a brand misses the critical element - the impact the word of mouth has. A better measure is based around how people act when they receive word of mouth (positive or negative) - how this changes their propensity to purchase
Anybody can be an advocate and pass on positive word of mouth. It’s more important to activate the whole user base than to try to find a particularly segment of advocates
The world is just more positive than negative - that’s the way people think about things. So you should expect more positive word of mouth than negative
One size doesn’t fit all - people are just more likely to talk about some products than others. Recommendations are more important for a dentist than they are for a supermarket. So you can only compare brands at the category level; indeed a different measurement tool may be needed by category
The latest edition of the 
The answer to this question is simple and complex. The simple answer is that you make sure you establish a set of criteria when you launch the community and then judge success against these. The complex answer come with some of the ways in which you may judge success.