Randall Farmer was one of the first community managers, before it was even a “thing” and long before it was a “hot job”. This experience, along with his intense interest in furthering our craft, means he has a lot of great knowledge to share.

One of the most fascinating things I’ve ever learned from Randall was about 5-star rating systems and the J-curve. When he worked at Yahoo! (back when working there was a “hot job”) they implemented 5-star ratings systems for Yahoo! Sites. They hoped this would help users identify the best sites and avoid the low-quality sites. Instead, this is what they saw:

With one exception, the average rating was 4.5 out of 5 stars. Wow, that’s great content, right? Nope. That’s the J-curve.

There are some major problems with 5-star ratings systems:

• Generally people only rate things if they love them or hate them.  If you love the dustpan you bought you might bother to go online & give it a 5-star rating. If it broke immediately you’ll angrily seek out the ratings system to punish it with a single star. But if it was just ok? You’re not going to bother adding a 3-star rating.
• People are lazy. If you’re highlighting the top-rated items on your site, you’re unlikely to ever get ratings on the other items. Most folks aren’t going to take the time to go rate items on page 2.

(The exception in the above graphs is Yahoo! Autos, because users were rating each other’s work. They knew these ratings would have an effect and keep the quality of the Autos community high, so they took the time to give accurate ratings. Yelp would be another good example of this.)

Takeaways:

1) Don’t use a 5-star rating system to try to determine quality of commodities. Try simpler systems, like thumbs up/thumbs down or positive-neutral-negative.

2) Try to find a way to get users invested in giving multiple, accurate ratings.