Tag Archives: metrics

What Community Professionals Need to Do to Weather These Hard Times

There’s no denying it: Community teams are being hit very hard right now by layoffs. The tech industry is seeing revenue dry up and taking massive action to reassure stakeholders, largely by letting people go.

We could debate the intelligence of these decisions, the morality of the choice, the side effects they’ll have to deal with. But that’s neither here nor there for Community professionals who worry they are next.

The fact of the matter is: companies are cutting heads and functions that they don’t think will help drive or retain revenue over the next 12-24 months. And if they’re cutting Community teams, that means we didn’t do a sufficient enough job generating and retaining revenue and/or we didn’t do a sufficient job promoting our success.


With that in mind, here’s what I see as the name of the game for the next year or two…

👇 Get Focused – no superfluous activities.

Community teams need to figure out where they can be most valuable and laser-focus on that. No unnecessary campaigns. No flights of fancy. No expansion of scope because some other Community team is doing it and it looks cool. Do 1-2 things, and do them well.

🔬 Experiment & Iterate – don’t launch & hope.

There is too much at stake to do a big launch and hope it works. New community ideas should go through testing & iteration, much like new product ideas. Start small, measure results, expand and pivot as needed from there. Don’t stake your whole reputation on a big launch.

📈 Measure Results – get as specific and close to dollar value as possible.

Fairly straightforward. We need to be measuring the value we’re providing, and tying it as closely as possible to driving or retaining revenue. Satisfaction scores and social reach are not going to do the job.

đŸ€ Build Connections – make community an invaluable partner for other teams.

Along these lines, the most defensible teams are the ones other teams want to defend! Don’t build off in a corner – find other teams that have challenges you can assist with. Become their hero, and they’ll become yours.

📣 Promote The Work – make sure nobody is unaware.

Community builders are givers, which often means they’re bad at self-promotion. You need to stuff that humble instinct in a trunk and lock it during this time. Every win you land should go to your boss, and possibly your boss’s boss, and possibly many other people in the company. You should be on the tip of everyone’s tongue. Reorgs and layoffs often, sadly, happen through gut executive understanding of what’s working. Don’t wait for them to ask if things are working – make it impossible for them not to know.

đŸ‘©â€đŸ”§ Share Stories – make sure people FEEL what the community has done.

“Evan, didn’t you just say metrics and dollars are the most important? What is this storytelling nonsense?” Yes, the numbers are the crucial thing. But storytelling is very powerful, and community generates amazing stories. Let the impact of your work be seen in the metrics and felt in the human stories you tell.


 

None of this is any guarantee you or your team won’t face cuts, of course. But these are the table stakes. Any Community teams not taking the above actions are at high risk until we get out of these dark waters. Stay vigilant, stay true to yourself, and hang in there – we will get out the other side, but it’s going to be rocky for a bit.

Measuring Community Value for Members: NetPromoter Score or Product Market Fit?

Measuring the value your community is providing to members is incredibly important. Without value, people won’t stick around, contribute, or spread the word.

So how do you measure this?

five stars on a blue and red piece of paper

No One Metric to Rule Them All

First, a caveat: you should never rely on a single metric for anything. Single metrics give you one angle of a situation that is complex. You should always compliment your primary metric with other viewpoints.

For example, if someone says that the community provides them value but you see incredibly low contributor rates, that could mean your community is riskily balanced on the backs of a very small number of contributors. Or you see great contributor rates, but when you review the quality, 90% are spam.

Choose the most useful primary metric, then supplement with other viewpoints.

The Case Against NetPromoter Score

A man staring at a laptop with his head in his hands, looking frustrated

NetPromoter Score (or NPS) asks what seems like a simple question:

“How likely is it that you would recommend [Organization/Product/Service/Community] to a friend or colleague?”

Members then answer by choosing a number between 0 (not at all likely) and 10 (extremely likely).

You then break members into different groups based on their score:

  • 0-6: Detractors
  • 7-8: Passives
  • 9-10: Promoters

You then subtract the percentage of Detractors from the percentage of Promoters to get your score.

This is a good overview.

NPS is rampant across the business world. It’s ubiquity suggests that it simply must be a high quality, useful metric. And although I’ve used it before, I have come to the conclusion that it’s generally problematic, and specifically a bad fit for communities.

0-10 Scales Are Confusing

Quick, tell me what a 6 out of 10 likelihood to recommend to a friend means!

Hard, right? The 0-10 scale for NPS is overwhelming for both customer and analyst. Research validates this: one study showed that a 5- or 7-point scale was better.

(They also found that there wasn’t a single best predictor, and that different question response combos resulted in different results – see my point above about looking at more than one metric.)

It’s Not Entirely Clear There Is Sound Logic Behind the Calculation

This piece nicely illustrates exactly how wacky this calculation can be:


“Let’s say we’re having a bad day and 10 respondents give us all zeroes: 0, 0, 0, 0, 0, 0, 0, 0, 0, and 0.
The average of these ten numbers is a 0. (Makes sense.)
NPS is -100. That’s the worst it can get.

Now, let’s say the team works really hard. They make the product so much better.
After all this hard work, we get all sixes: 6, 6, 6, 6, 6, 6, 6, 6, 6, and 6.
The average of these ten numbers is 6.
But NPS is still -100.”

This is not only depressing, not only confusing, but also doesn’t give you good directional feedback. You just increased likelihood to recommend by a massive amount
yet your score says you made no progress. This could lead your team to abandon the work that got you those 6’s and go back to the drawing board, even though you were on the right track.

There Are Reasons Someone Might Not Recommend A Community They Love

This last point comes from personal experience. I’ve used NPS many times in my career. And every time I dig into the details of the responses, I see oddities based on the exact phrasing of the question. Common ones:

  • “I wouldn’t recommend this to my friends because I have no friends.”
  • “I wouldn’t recommend this I don’t ever recommend products.”
  • “My friends don’t want me to bug them with product recommendations.”

With communities, it becomes even more interesting:

  • “I wouldn’t recommend this community to my friends because I don’t want them to know what I’m posting here.”
  • “I wouldn’t recommend this community because my friends aren’t into this topic.”

So your member could be experiencing your community as the brightest point in their life, the reason they get out of bed in the morning, and yet they’d give you a bad NPS. Again – this is not helpful directionally.

The Case for Product-Market Fit

Green apple with measuring tape wrapped around it

The Product-Market Fit (PMF) survey asks:

“How would you feel if you could no longer use [Organization/Product/Service/Community]? A) Very disappointed B) Somewhat disappointed C) Not disappointed”

Then you simply report on the % of members that fall into each category.

There’s a nice overview here.

Now, to be clear, I’m not 100% sure PMF is the right solution, and I have limited personal data points. But on the surface, it feels like a better fit.

It’s Simpler

Three options. Worded in a human way. Far easier for the member to determine, and far easier for you to interpret; “Hey, most folks would not be disappointed if we shut this community down” is a pretty definitive statement.

It Focuses on the Member

Rather than focusing on recommendations, which I’ve already demonstrated are highly complex things, this question focuses purely on the member’s experience with your community.

It’s Encompasses Value and Emotion

Especially with communities, you want to provide value but ALSO be eliciting a strong emotion. Someone might find a community useful but not be passionate enough about it to contribute. This question nicely straddles the usefulness of the community and how strongly the member feels about it.

It’s a Familiar Term for Most Startup Leaders

The hardest thing about moving away from NPS is that everyone seems to use it, so you’ll get skepticism when you say you want to try something new.

Thankfully, entrepreneurs talk about product-market fit constantly – so you’ll still be speaking their language.

The Magic is in the Responses

Regardless of which survey you use, be sure to include a “Why did you give that score?” write-in question after the numerical rating. Yes, having a score and being able to look at which way you’re trending is valuable. But more valuable is knowing WHY people feel this way. This is what you can spin into gold. This is what is really bugging or delighting people.

So even if you’re stuck with NPS, make sure you get those write-ins. Ultimately, what your members say is more valuable than any number.

Keep Experimenting

My goal here isn’t to get you to adopt PMF, but rather to carefully consider the value of NPS for your community and to explore alternatives.

As mentioned above, no one metric will serve all your needs.

No community is the same, and you may have unique aspects that you should measure. Don’t get bullied into using others’ metrics unless they’re actually valuable for you.

Try different things. Try validating them – do you actually see recommendation actions from high scorers? Increased contributions? Or are people saying one thing and doing another?
And lastly, please share – we’re all traversing these challenges together, so I’d love to hear from you about what worked for you!


Stars photo by Towfiqu barbhuiya

Frustrated man photo by Oladimeji Ajegbile

Apple photo by Andres Ayrton

Events drive retention

The effect that user groups has on sales and retention is undeniable. Over the past five years at Brainshark, we have carefully watched the effects of meeting our customers for these in-person events.

Here are the results:

  1. Within 90 days of the meeting, our customers use the product an average of 15% more.
  2. Renewal rates are 10-15% far higher for customers that participate in the user group program.

From CMX Hub

I am so ecstatic that community professionals are beginning to measure and share numbers like this. I will keep yelling until I go hoarse: Community management is about retention! It’s a hard thing to measure, but it is measurable, as demonstrated here. CEOs are not great at understanding this, and you’ll often get pushed to focus on lead-gen or upselling. These are not the jobs of community, and if you try to build a community while pushing your product, you will face significant challenges. Community management is about retention.

February Community Manager Breakfast Notes – Metrics, Offline Community, and more

At February’s San Francisco Community Manager Breakfast, we eschewed the pre-set topic and chose topics as a group. The result was a fantastic, varied conversation with folks from all different experience levels, business types, and focuses. Although you won’t get the full context from the notes – you’ll have to come to breakfast for that – there are some great observations and suggestions below.

A huge thank-you to Meredith Black for taking the notes! If you’re looking to hire someone very intelligent with events skills, check out her LinkedIn!

1. Launching a community from scratch

  • Choosing community focus
    • Test with Minimum Viable Communities – do things as simply as possible (Facebook groups are easy) and see what sticks. Less risk.
    • Consider that you may have more than one community – especially if you’re a two-sided marketplace. Don’t treat them the same.
  • Research
    • Go to Twitter chats, forums where market exists.
    • Hang out, follow, engage in conversations.
    • Note what engages people, where gaps are.
    • Once your community has started, these places can be perfect for sharing about your CMTY organically.

2. Engagement

  • What is a real, loyal CMTY member? Sticky, engaging, and offering value.
  • Do user testing for ways to push interaction.
  • ID the evangelists (Customer Support can be a great source):
    • Figure out how you can help them.
    • Give them responsibility – they want it, and it’ll help you.
  • Personalize:
    • Be the face of the brand: sign social media posts with your name, be the face/voice of the brand.
    • Use a personal email (ie Shannon@monument.com) – if you can’t handle the volume, have the rest of your team help with it.
    • Do the things that aren’t scalable (a la Paul Graham)
      • Phone calls, emails, friendships, 1-on-1 asks

3. Platform

  • Hard to launch a CMTY without a platform/ways for members to communicate.
  • Facebook Groups definitely work – but FB has a tendency to interrupt/pull functionalities. Move off it as soon as you can.
  • Platform suggestions:
    • Mobilize (built by former CMTY mgrs.)
    • Jive (can segment, has gamification)
    • Mighty Bell
    • Discourse
  • Mobile community platforms still pretty rare.
  • When moving a CMTY from one platform to another: do it in buckets, introduce users to forum, measure activity.
  • Moving has risks, challenges, so it’s necessary to get the CMTY more engaged.
  • Platform architecture can be overwhelming – don’t underestimate.

4. Offline CMTY-building

  • Offline is a trend (vs. 4 years ago).
  • Development is the same (set the tone/rules, power-user program, scale it).
  • How do you find your initial members?
    • Relationships are built face-to-face: get out there, tailor, make it personal.
  • Collaborate/empower users so they initiate events for the brand.

5. Offline Metrics

  • Know what the actual company goals are (often, management isn’t sure):
    • Brand recognition/association
    • Member-to-member interaction
    • Retention
    • Goodwill
    • Etc
  • Don’t have ROI measured yet? Provide management/C-suite with tons of general data:
    • Activity level
    • # signups
    • Engagement volume
    • Etc
  • Tell both stories – metrics and personal:
    • Emotional: interviews, feedback, Yelp reviews, etc.
  • Share successes pre-emptively:
    • Data
    • Learnings (shows you’re not just flailing)
    • Roadmap that can be quantified
  • These are the same challenges as for other soft departments (like PR).
  • Tools:
    • Google Analytics
    • Sprout Social
    • CRM
    • Good ol’ spreadsheets

6. CMTY+ (cross-functional integration)

  • Make friends internally and externally – get buy-in of tech team, C-level, support, finance, etc.
  • Don’t reinvent the wheel – partner instead.
  • CMTY+Sales:
    • Community can help retain, make repeat sales more likely.
    • Leads are more qualified/shared.
    • Deals close faster.
    • Benefits maybe aren’t apparent through regular CRM data.
  • CMTY+Marketing
    • Leverage current customers for leads to new growth.
    • Track evangelist movements, put in a bucket, use for PR/marketing (collateral, landing page quote, great story, reference for potential investors, etc).

Hope to see you at the next breakfast!