Internet ratings get only three stars from me

Receive free Technology updates

We’ll send you a myFT Daily Digest email rounding up the latest Technology news every morning.

There are three kinds of lies on the internet. Lies, damned lies, and one-to-five-star ratings.

I’m not just talking about the kind of ratings that can be bought from a fake reviewer for $5 a pop (though those are indeed a plague). I’m talking about the supposedly real ratings that we give by mindlessly tapping our thumbs on all five stars when we are prompted to review yet another nausea-inducing cab ride or mediocre fitness class — not because this is a true reflection of our views but because it’s the fastest, most frictionless and guilt-free option.

Online reviews depict a world in which everything seems to be wonderful apart from when it’s abominable. Ratings have a “J-shaped” distribution: a large number of top scores, some rock-bottom ones and almost nothing in between. Two- and three-star reviews are virtually non-existent. A study published in Nature in 2021 found that more than 80 per cent of online reviews came with four- or five-star ratings, creating a “positivity problem” that made it virtually impossible to discriminate between products and services.

My most recent run-in with the positivity problem came on a recent trip to the US, where I stayed in a “luxury” Airbnb apartment that had an almost perfect 4.85-star rating from 115 reviewers. That was despite an almost intolerable sickly sweet odour inducing suspicions of a cover-up for something grim; the fact that the coffee table books displayed in the photos were cardboard fakes; a bedroom blind being broken; and the gym having no air conditioning while outdoor temperatures were above 40C.

“Thank you for your visit, we will be giving you a 5 star review across the board! We are glad you enjoyed your stay :)” This rather passive-aggressive, emotionally blackmailing review-nudge came just minutes after I had checked out.

If brave enough, I would have given the apartment three stars. But I wimped out and gave no rating at all. The system is not geared towards honesty. Had I submitted a poor rating, I would have had to face the embarrassment of a public review that makes me look like a nasty or strange person (what kind of monster gives three-star Airbnb reviews?) plus I would have had the guilt of potentially damaging the host’s livelihood.

And that brings us to a crucial problem. Online ratings are now so important to the sellers of the products and services being reviewed, as well as the platforms that host them, they can no longer be relied upon. It’s Goodhart’s law of economics at play: once a measure becomes a target, it ceases to be a good measure.

If businesses are not buying fake reviews, they are encouraging us to leave positive ones. A company I bought a pair of sandals from last month has sent three “friendly reminders” to leave a review, offering €10 off my next purchase — the five-star option is helpfully preselected. On Airbnb, the words explaining what each star rating means nudge strongly towards the top: the bottom one star is “terrible” but five stars is only “great”. Presumably anything more enthusiastic (“Superb”? “Wonderful”? “Amazing”?) would discourage users from selecting five stars, giving the platform fewer of the top ratings that boost sales.

“Because these ratings have become the most important source of information for consumer decision-making, it’s become this huge edifice now that’s managed on all sides,” says Philip Fernbach, co-author of The Knowledge Illusion, a cognitive scientist and professor of marketing at the University of Colorado, Boulder.

Fernbach co-authored a 2016 paper that found product ratings on Amazon had very little correlation with the quality verdict of well-established metrics, such as by Consumer Reports, a US non-profit specialising in scientific product-testing. He and his colleagues are conducting research into whether reviews rated “helpful” are predictive of quality. The initial findings? They are not: reviews are judged helpful because they sound authoritative or use technical language, not for accuracy. So even the online ratings system for online ratings appears to be broken.

The online ratings system is a reflection of an internet that rewards extreme points of view and punishes uncertainty. The online ratings system is a reflection of that. Opinions are distributed along a J-curve, only in reverse: everything is either terrible or amazing, but mostly it’s terrible. Nobody wants to hear your “on-the-one-hand-on-the-other” takes; they want savage takedowns or, failing that, fawning praise. It keeps you engaged. The problem, of course, is that the truth often lies somewhere in the messy middle.

[email protected]