Tracking Down Online Community ROI (Part 2: Business-Side Metrics)

by Matthew Lees

Part 1 looked at community-side metrics. This is the data you get from your community’s Web server log files, your community platform database, and any third-party analytics systems (such as Google Analytics or Omniture) that you’re using.

It’s also the data that you – as a business sponsor, community manager, or other stakeholder – likely have direct access to. And while it’s important information, it’s used primarily to help ensure the health of the community, not quantify and provide insight into business value. For that, you need to tap into business-side metrics.

Business-Side Metrics
These are the metrics that do show business value. Unfortunately, for most online community use cases, such data lives in places that you probably don’t have direct access to or control over. This is where the legwork and the relationship building that Mike Rowland referred to in the previous post come into play.

Where to look in your organization and who to build relationships with depends on what you’re after. Here are four common business cases for B2B communities, with an overview of their potential business value as well as mention of the relevant business-side metrics, location of these metrics, and people who can help you access and understand these metrics and what they mean for the bottom line.

1. Service and Support. Reducing contact center costs is one of the primary business goals of a community in which customers help answer each others’ questions and solve each others’ problems (via what’s often called “peer-to-peer support”).
Business-side metrics: number of incidents (by source, e.g., phone, email, chat, etc.), first-contact resolution, agent hours
Where the metrics live: contact center analytics system
Who to make friends with: not only the VP of Support, but also the manager who is the most fluent with the call center’s reporting and analytics

2. Product Development Feature Set and Road Map. Here you’re probably looking for (a) ideas for new products and services, (b) ideas for new features and functionality, (c) ideas around improving customer-facing processes (i.e., making it easier for customers to do business with you), and (d) the prioritization of these ideas. These ideas and their prioritization by customers can improve processes, reduce time to market, and give you higher confidence that your product road map is what your customers want.
Business-side metrics: number of customer ideas that are implemented; number of existing ideas that were validated by customers; time to market; dollar value of reduced time to market (can be a squishy number)
Where the metrics live: product tracking system; business process systems (ideally these all track the sources of ideas)
Who to make friends with: product development / R&D teams, particularly the keepers of the road map and features/capabilities lists

3. Customer Acquisition and Lead Generation. Communities are a great way for people to go beyond what they read on your Web site and in your marketing collateral, to get a sense of how people are using your products in the real world. So prospects are part of the community ecosystem as well as existing customers. A vibrant community full of helpful, engaged customers can be effective in moving prospects into your sales pipeline.
Business-side metrics: number of new accounts that came in through the community, new revenue from these accounts
Where the metrics live: CRM system or other sales tracking application
Who to make friends with: the sales team, particularly the sales operations manager who tracks sourcing

4. Customer Satisfaction and Loyalty. Numerous studies have shown that online communities can have a positive affect on customer satisfaction and loyalty. The tricky thing in demonstrating this for your own community is to separate out cause and effect. Communities can be self-selecting; your most satisfied and loyal customers are probably over-represented in your community. For them, the community didn’t cause their high level of satisfaction, for example. Any surveys you do to measure satisfaction and loyalty should take this into consideration.
Business-side metrics: survey results; customer satisfaction / loyalty methodology or system, such as Net Promoter Score (NPS)
Where the metrics live: survey results database; satisfaction, loyalty system
Who to make friends with: the marketing specialist who measures customer satisfaction and loyalty for your organization

Legwork and Relationship Building
You may have noticed that the business-side metrics are really just the ones that your organization and your colleagues are already using to identify and analyze business value. You’re just looking to apply and tune them towards quantifying their impact from the community.

Of course, while the methods may be familiar, it isn’t necessarily easy to compile metrics and estimate dollars saved and/or generated. A lot of it comes down to doing the legwork and building relationships with the right people. Ideally determining community ROI is at the top of their priority list as well as yours. It will take time and attention to come up with ROI hypotheses, test them using data you’ve tracked down from wherever it lives, analyze the results, revise your hypotheses accordingly, and iterate. Hopefully your colleagues become partners in these efforts.

So how do you build these relationships, make those allies, and get the information you need? We’ll leave that for another day. But experience shows that chocolate helps…

Determining Online Community ROI (Part 1: Community-Side Metrics)

by Matthew Lees

In recent months my research, writing, and client work has (happily) focused on one of the hottest topics in social media…calculating ROI. For a variety of reasons, determining quantifiable ROI of social media programs and online community initiatives presents a variety of challenges. There’s nothing new in this statement; people have been trying for years to make it easier to figure this out. But ROI keeps fighting back. It’s getting to the point, though, where business executives will be expecting a more quantified understanding of the impact customer communities, for example, are having on the bottom line. Knowing that it’s “the right thing to do” from a customer-centric perspective isn’t going to cut it much longer.

One of the challenges is the fact that some (or even most) of the information needed to measure ROI isn’t in a convenient location.

Last year in a post called “B2B Communities – What Works,” Mike Rowland discussed a handful of essential best practices for B2B communities. He wrote: “You can measure the ROI for B2B communities, but you cannot get there by using only community software metrics and/or web analytics packages like Omniture or Google Analytics. None of these provide true value metrics that have an economic value associated with them. To get to ROI, you must build relationships within your organization so you can obtain real data on customers, leads, ecommerce transactions, etc.”

Right on, Mike. To get at community ROI (whether for B2B, B2C, or any other type of community), you’ve got to track down data from several sources – particularly from sources that you probably don’t have direct control over or access to – which takes building relationships, making allies, and a bit of legwork.

Community-Side Metrics
The relatively straightforward part of the legwork compiling the data you do have direct access to. This is the information you can get from log files, your community platform database (often through an analytics dashboard), and, if you use one, a third-party analytics package (such as Google Analytics or Omniture). The types of things available from these community-side sources include:

•    Traffic and Usage. Pages served, page views, visits (and unique visits), time on site, etc.
•    Membership. Total members, new members, active members, reputation and ranks, etc.
•    Activity. Posts, comments, ideas, invitations, votes, ratings, subscriptions/notifications, downloads, views (e.g., of video clips), time between posts, etc.
•    Search. Both quantity of searches and specific search terms (such as top 20 search terms)
•    Other. Moderation (e.g., moderator touches), referrer pages, etc.

Ideally you’ll be able to break down this data based on important parameters, such as time and location within the community. For example, you’ll probably want to look at the all-time number of members, as well as the number of members over a given time period (typically week-over-week or month-over-month). And you may want to look not only at total pages served within the community, but also at pages served within particular areas (such as forums, blogs, idea sites, and so on).

Community-side metrics can be very useful, but, as Mike wrote, they don’t illustrate economic value. At least, not in and of themselves. They’re certainly useful in terms of knowing whether or not the community is healthy (three consecutive months of decreasing page views, for example, might tell you that something is wrong), but they only get you part of the way toward determining ROI.

Up next…building relationships and doing the legwork to identify the business-side metrics that get you the rest of the way there.

Social Media Metrics – Driving to Value

We’re members of the Online Community Research Network and recently received the latest report on Community Metrics derived from a survey of the membership. While we’re happy to see a lot of progress in the responses about tying measurement to business objectives, we continue to see confusion about measuring value. Over the past ten years, we’ve developed a methodology that can help online community and social media managers structure their reporting in order to focus on the value their efforts produce in terms of business objectives.

Looking  at the report’s question #12 (Were your community’s metrics created in support of your organization’s broader business goals or were they created independent of a corporate business?), the following responses were given:

• 47% Created to support existing business goals
• 31% Created independently but helping refine existing business goals
• 22% Neither of the above (summarized from three additional responses)

Looking at what metrics the respondents use to support or refine existing business goals provides insight into the confusion over what constitutes value in online community and social media efforts.

Question 19 asked “What are the three most important community key performance indicators (KPIs) in the reports you send to upper level management?” The answers are a startling contrast to the answers to Question 12:

1. Number of Page Views or Clicks
2. Number of Site Visits
3. Number of Unique Visits

Why are these responses startling? Because the metrics are traffic metrics not value metrics. These are base level metrics not KPI worthy metrics for upper level managers. (In fact, three of the top five metrics measured as detailed in an earlier question were traffic metrics too: Unique Visitors, Page Views, and Visitors. Only two were not: Registrations and Posts.)

What these two questions’ responses demonstrate is that the respondents are still struggling with determining value from their community work that truly builds into measurable business objectives.

When asked about ROI, 71% of respondents confused engagement and traffic metrics with value. Only 29% correctly identified a tangible value metric to use in measuring ROI.

To provide a little clarity in reporting metrics, let’s look at how Impact Interactions’ reporting methodology can help. First, our categories are structured as follows:

• Traffic - The basic building blocks that measure “How Many?”
• Behavior – The second level of metrics measuring conversion and engagement
• Value – The highest level of community metric where the activity has an economic or dollar value associated with it (This is what management really cares about!)

Some of the actual metrics that we use for clients are as follows:

1. Traffic - Unique Visits, Unique Visitors, Page Views, etc.
2. Behavior- Page Views/Unique Visit, Page Views/Unique Visitor, Active Members/Unique Visitors, New Registrations/New Unique Visitors, Total Registrations/Total Unique Visitors, Downloads/Registered Member, Content Added/Registered Member, Content Added/Unique Visitor, Downloads/Unique Visits, Full profiles completed, Referrals from Twitter/Facebook/YouTube, etc.
3. Value – Number of successful customer support resolutions in the community, Total Contact Sales Inquiries/ Total Unique Visitors (or Registered Members), Total Leads Qualified/Generated, Product Referrals, Positive Product Reviews as a % of Total Product Reviews, Direct Revenue Generated from Community Activities, Length of Sales Cycle for community member vs non-member, Average Purchase Size/Frequency for community member vs non-member, etc. 

Take a look at those metrics again. The first two categories of Traffic and Behavior can usually be obtained using the platform’s tools (like Jive, Telligent, or Lithium) or through your web analytics’ tools (like Google Analytics, Omniture, or WebTrends). The Value metrics take a little more work. In fact, to really be able to perform a realistic ROI calculation, you will need to get help from outside the community/social media area of your organization.

To derive an ROI related to marketing objectives from a community, you’ll need to access your CRM system. For a support ROI, you’ll need to know the cost per interaction in complementary/competitive areas such as a call center. The standard tools won’t get you there, you’ll have to build relationships within your organization in order to really build a solid analysis that ties back to business objectives. An ROI model built on traffic will contain far too many holes to be useful.

We’ve been helping our clients with these issues and have developed a strong set of best practices that can help you succeed in your reporting. Please feel free to share your insights into this issue and ask questions about reporting and analyzing your community and social media efforts. We’re happy to answer them and help reduce the confusion.

2009 Trends — It’s Not All Doom & Gloom

It has been a very busy summer here at Impact Interactions. We’ve added more work with our existing clients like Cisco and SAP, added new clients, added more staff, launched a new small business service (Impact Social Media), and have received many more calls about our services. Taking a step back from the activity to analyze the social media/online community industry, we’re seeing the following trends emerge:

1. While enterprise level organizations are being very careful about spending money, there is a lot of interest in the social media area. We are seeing budgets freeing up, new projects starting and excitement about using social media to market products and services.
2. Enterprise online communities continue to launch at an amazing rate. We haven’t seen this type of growth in corporate sponsored B2B communities since 2000 when the concept was very new. The difference is that now there are third party sites like Twitter and YouTube to integrate into the communities.
3. Every one wants to avoid making mistakes, placing real best practices at a premium for enterprise companies. While there is always some learning by mistakes made in any venture, the companies we are dealing with always tell us that they want to avoid making the basic mistakes that others have made.
4. Analytics are at a premium, but not understood very well by some marketers running communities. We continue to see interest in base level metrics around traffic and basic engagement, but less understanding of value. Part of this is due to the over-reliance on Google Analytics as the main tool instead of a more powerful solution like Omniture. Google Analytics is a basic tool not a true enterprise level analytics tool in our opinion and experience.
5. In conjunction with number 4, we also see clients and prospects changing platforms in part due to poor reporting and administrative control pages. Many vendors seem to put reporting and analytics into their platform as an afterthought. Platform providers moving sharply ahead of the field in providing reporting and analytics for their software are Telligent, Lithium, and Jive to a certain degree.
6. Outsourced providers of social media expertise and management in areas such as moderation, social media monitoring, reporting, and integration are gaining more interest among enterprise level companies. With headcounts frozen or worse, organizations are looking outside their company for experienced help at an reasonable cost.
7. Lastly, the social media consulting industry remains very fractured. There are simply too many small businesses, individual consultants, and former software personnel chasing deals resulting in lower pricing and no concentration of expertise in a meaningful way. In other words, this industry is ripe for a consolidation play. This is what Jeff Dachis of the Dachis Corporation in Austin is slowly building towards. We think that there are multiple opportunities for consolidation and are actively looking for non-software companies to acquire or align with to gain a larger share of this growing market. (It’s only a matter of time before the big guys like Accenture, IBM, or big advertising agencies buy up the industry’s expertise to consolidate their market share.)

So far, 2009 has been far from the doom and gloom year that most were predicting in our industry. Certainly there has been some shakeout, but overall 2009 is shaping up as a really good year overall for social media and online community service companies.

Do you agree with these trends we’re seeing? What else are you seeing in our industry? Please share your comments below.

Mike Rowland, President

Online Community – Moving Beyond Metrics to ROI

We gave a presentation on building ROI models for online communities at the Online Community Unconference in Mountain View, CA on June 10th. It was added because so many of the participants stated that ROI and calculating the value of their community was the most important issue they faced.  So, we didn’t have time to build a true presentation, but rather lead a workshop for participants to learn more. It was lead by our president, Mike Rowland.

Here are the summary notes taken during the session:

• Have to first identify what is the economic value of the community to the organization offering it: (Don’t confuse traffic or behavior metrics with value)
  • Cost Avoidance
  • Increased subscription rates or lower churn rate
  • More frequent purchase rates
  • Higher purchase level/amts
  • Faster close for large item sales
  • Reduce lead generation cost
• Once you’ve identified your value metrics, break down your metrics into 3 buckets to look at communities:
  • Traffic – PV, visits, visitors, etc.
  • Behavior – What they do when the get there, who they are, download/visit, contribution/member, responses by employees vs. customers
  • Value – can attach an economic value to it. Need $ to get to a true ROI model. See above list.
• You have to build relationships w/ the folks in your company. Need access to other systems. You cannot build ROI from community analytics provided by software vendors or from traffic and behavior metrics alone. 
• ROI Frameworks:
  • Cost Avoidance
    • The person who proposes the question needs to verify the answer. This is a feature needed in the platform.
    • # of community support resolutions X $ complimentary support resolution (1-800 number) = total cost avoidance -> economic value
    • Track over set period of time (e.g. quarterly or yearly)
    • ROI = (total economic value – total costs to set up and run forum) / total costs –> over one period and as a percentage
  • Increased subscription or reduced churn
    • Customer database compared to community database
    • cust. database = Average churn rate (e.g. the number of months the avg user subscribes) X price/subscription = customer value
    • Community database – look at active members and see if the churn rate is better or worse.
    • Price will be the same, so you’ll have to see if the churn rate was more or less. If subscriptions are longer, than you have a higher customer value for community members.
    • Shows you slowed the churn rate down.
  • More frequent purchase or Higher Purchase level/amts
    • Use your eCommerce DB or CRM system
    • What is the avg amt customers spend/purchase?
    • go back to comm DB and parce out active members and compare to ecommerce DB (which ones spend more/purchase?)
    • Do comm members have a higher spend/purchase? active comm users X avg $ they spend = economic value
    • Need to trend this and see how it fluctuates.
    • what is the average number of items in completed shopping activity? (e.g. 1.6 items) Do comm members buy more?
    • Avg cost/item X avg # items = economic value
  • CRM decrease cost
    • Want to find what avg value of customer is
  • Faster close of sale (Good for large purchases like software or hardware systems)
    • How fast are organizations moving through your CRM system to sale?
    • Identify active organizations in community DB to compare them to avg organizations.
    • How long does it take the avg. organization to go through sale stages? What’s the cost/sale? Do active organizations in your community go through more quickly and spend more?
  • Lead generation cost
    • Similar to above, but use cost to generate a lead for average customer versus those which originate in community/social media campaigns
• How can you tell if a user came to your comm and then bought your product through a 3rd party reseller? You can’t unless your resellers provide access to their databases which is next to impossible to get.
• Users of support communities become brand neutral after their issue becomes resolved.
• Hidden costs of community for ROI Analysis, make sure you count these:
  • Servers
  • development costs
  • customizations
  • widgets
  • maintenance
• Make sure that you are amortizing your costs across the same time period as your economic value or you will skew your results.

One point to remember is that the value of communities really is derived from active members, not all members. So define your active members with criteria that meet your behavioral key performance indicators (KPIs). For example, an active member can be someone who posted in a forum, downloaded a featured whitepaper, uploaded content, or viewed a webcast in the past month. For B2B especially, it doesn’t have to be an activity within the past week as most B2B community members average 2-3 visits per month unless they are deep into the sales cycle.

The number one issue to watch out for when building your ROI framework is the use of proxies. If you cannot get the data, don’t guess at a proxy for the value because the more proxies you utilize the bigger the house of cards that you build.

Lastly, value can be a set of different items. For a subscription based community value can be both churn rate differential and purchase levels. ROI is not a single value formula, it is a multiple value formula in most cases as there is marketing value in support communities and vice versa. So make sure that you are at least attempting to capture as much of the value drivers as possible in your analysis.

Want to learn more about online community or social media ROI? Contact us today or leave a comment.

Why Does Everyone Believe Community ROI Is So Hard To Measure?

Just finished reading Jeremiah Owyang’s blog on ROI for online communities and social media. The entry is here. Jeremiah’s suggestions are accurate and some of the comments from our colleagues are really helpful. Here is our commentary which we’ve also posted on Jeremiah’s blog:

What I find interesting about this topic is the general disagreement of the relevance of ROI in the discussions of community and social media experts. We’ve attended events, webinars, and industry meetings, where people were unclear or downplayed the importance of this measure for social media and communities. Others in the Social Media space blog frequently that ROI is either tough to do or not relevant.

At Impact Interactions, we believe that ROI is a crucial element of the community building and management process. You must start building your framework before the community even launches, then refine it over time. But you cannot use just the metrics from your community, you must align them with additional data from within the organization (CRM records for B2B for example).

For B2B support clients, we measure technical questions answered by members as a cost avoidance measure to demonstrate the scalability of the community versus call center costs. We supplement it with survey data on customer satisfaction, purchase influence, and information utility. It all adds up to a large ROI.

For a marketing focused B2B community, we built a framework that demonstrated the influence that the community had in influencing sales of multi-million dollar contracts. We mined the transactional data and compared it with the CRM records to develop a pattern of influence on sales velocity, lead generation, and sales.

For a B2C automotive parts company, we compared sales transactions from the companies e-commerce database with community transactions to find the ROI for the community. It also reinforced the powerful notion that community members were buying more frequently than non-community members and that each purchase transaction was larger than those of non-community members.

For a B2C subscription based service, at Participate.com we demonstrated that community members churned at a rate 50% lower than non-community members, resulting in millions of dollars of revenue and profits.

Each of these clients had an ROI on their community of over 100% once their communities scaled.

It is not hard to develop the ROI framework, but it does take time and relationships within the organization to get the appropriate data. If you are a community manager, you need to build a network outside of the community area in your organization in order to align the community’s analytics with your organization’s focus and goals. Only then will you be able to tap into CRM or e-commerce databases to validate your framework.

We have some basics on B2B ROI in presentations available for free in our Social Media Resource Center on our website. Please feel free to visit and download the presentations. In our introductory deck on Impact Interactions, we have quotes from Cisco, Mercury Interactive, and ATT on their ROI from their online community efforts. Here is the link: http://www.impactinteractions.com/social-media-resource-center.html

What Jeremiah has posted is absolutely spot on. But is up to you as a community manager to act. In this environment, you cannot afford to have your community (and job) viewed as a soft application that doesn’t produce tangible, measurable results. If you’d like to learn more, please contact us.

Impact Interactions Strengthens the NetApp Online Community

NetApp launched its first online community in March 2008 to build better relationships with customers, partners, prospects, and other stakeholders. Initially, the community was well received, but NetApp’s community team looked to Impact Interactions to accelerate their results. Working with prominent technology companies such as Cisco, SAP, Intel, and HP has given Impact Interactions the knowledge and understanding needed to guarantee that NetApp avoid many of the pitfalls that newer online community teams make.

“Our initial assessment of NetApp’s online community determined that their community team was using many of the correct tactics for building a successful community,” said Mike Rowland, President of Impact Interactions. “But with our strong experience in B2B community creation, management, and measurement we were able to provide an enhanced roadmap for NetApp based upon proven B2B best practices.”

Working together with experienced consultants like Impact Interactions can give an organization that is new to online community a significant advantage over competitors who either take on the task alone or avoid it altogether. These advantages include a shorter development stage; generating realistic, measurable objectives; and avoiding costly mistakes. Many communities don’t understand how to increase conversion and engagement rates or how to generate useful metrics reports for analysis, which Impact Interactions can help with significantly.

The advantages of using an outside consultant aren’t limited to those who have yet to launch a community. The fresh perspective and experience they provide can benefit a community at any point in the lifecycle. This is partly because public information is still scarce on the development and management strategies needed to make communities a successful endeavor. Furthermore, most of the research and information that is available about online communities is based around business-to-consumer (B2C) communities. The strategy and techniques used for B2C communities is often times very different than what should be used to make business-to-business (B2B) communities successful.

This is an important concept to grasp when developing a community. “A lot of times I hear people say that they want the community to be like Facebook or MySpace, but what they don’t realize is that social networks and B2C communities have a totally different purpose and a totally different audience than B2B communities. The strategy and management techniques used for these professional communities need to reflect these differences,” said Mike Rowland, President of Impact Interactions.

For additional information regarding online community consulting or moderation services, contact Mike Rowland or visit http://www.impactinteractions.com/.

Online Community – B2B Return on Investment (ROI) During Recessions

As we move through a downward business cycle, smart companies are moving counter intuitively to engage more with their customers rather than decreasing their online community efforts. The slide below from a presentation during the height of the Dot Com implosion in 2003 shows that even in a recession, online communities still deliver impressive returns on their investments:

What is interesting about these quotes is that the communities referenced were launched in the 1998 to 2000 time frame when online community was a new offering on the web. To gain these results, our clients had to change the behavior of their potential members to include online communities in their work routines. While it seems old fashioned today when everyone knows what an online community is, these results demonstrate one of our best practices:

Too often organizations buy into the idea that the offering itself will be enough. The “build it and they will come” idea has been proven to fail almost every time. And yet, we see organizations thinking about cutting their community investment or launching a community and letting the community run itself. That is a recipe for failure as well. It also reflects the thinking that online communities and social networks are a ’soft’ application (meaning that they cannot prove their value quantitatively). It is for this reason that many managers move to cut online community budgets dramatically during downturns. This is exactly the wrong time to cut, this is the time to build.

In times of economic uncertainty, your customers and prospects will perform more due diligence than ever prior to buying your product or service. Online communities and social networks are valuable tools in that process. Will a community without a strong company presence be able to influence the purchase decision? Probably not. This reinforces a second best practice:

So how does this tie into the ROI argument? Simple… if your organization is not prepared to engage with the community members by answering their questions, providing documentation, linking them with content, and asking their opinions your community will wither away during the downturn. If your organization is ready to engage with your community, you will build additional brand loyalty among members. Your community will be vibrant in comparison with your competitors who are pulling back. You will be demonstrating the value of working with your organization over the competition. By providing correct information rather than partially correct information (community managers facilitating conversations do this part), you help your prospects learn more about your products/services/company and build confidence in them that you’ll be there to help.

Remember that in most B2B communities, the ratio for those visitors who read versus those who add content is roughly 25 to 1. This means that every successful interaction potentially influences another 25 visitors on average. Depending upon the product or service, it can be much higher. Influence is correlated with purchase intent. When your community is functioning properly and is well managed, the influence it wields is tremendous.

By mining your registration database against your customer database, you’ll be able to prove the value of your online community. By using a proxy of 2-5% of sales made to community members were influenced by the community, you’ll be able to calculate a basic ROI. To refine the proxy, survey your members about the influence the community has had on their purchase decisions. You’ll be surprised how large a percentage of community members have been influenced by something they read or received from the community. But if you are not investing in your community, you will not get the growth needed to generate a positive ROI.

And that’s just the sales/marketing ROI. For those B2B communities engaged in self-service customer support, the cost avoidance equation is easier to figure out and measure.

Want to learn more? Visit our website (Impact Interactions) to download our free presentations on online community best practices.