Can You Trust Online Reviews?

For Mandana Yousefi, the road to becoming an online critic started with chicken fried rice. Yousefi had ordered takeout from a Chinese restaurant in Washington, D.C., and was about to dig in when she noticed four long strands of hair in her food. She called to complain and demand a refund, but the restaurant insisted that she order something else—and tried to charge her $1 more for the food she selected. As a “creepy side note,” Yousefi says, she was asked to return the fried rice.

Outraged, she headed to review site Yelp.com and slapped the place with one star out of five. “They had gotten away with being bad, and I wanted to give other people a heads-up,” says Yousefi, 29, an analyst for the U.S. Department of Agriculture. Since then, she has written more than 320 reviews on Yelp, covering everything from a local jewelry store (“I adore this business”) to a nightclub in Santa Rosa, Calif. (“rudest door staff and incredibly grimy crowd”).

Whether you’re looking for a place to eat or a handyman to fix a faucet, the business has probably been picked apart by a citizen critic like Yousefi. That promise of honest, trustworthy opinions, democratized by the Web, is what lures millions of visitors to Yelp, as well as sites such as Amazon, Angie’s List and TripAdvisor. Yet with ratings now popping up just about everywhere online, you might wonder how much faith you should place in the wisdom of the masses.

From just $107.88 $24.99 for Kiplinger Personal Finance

Become a smarter, better informed investor. Subscribe from just $107.88 $24.99, plus get up to 4 Special Issues

CLICK FOR FREE ISSUE

Sign up for Kiplinger’s Free Newsletters

Profit and prosper with the best of expert advice on investing, taxes, retirement, personal finance and more - straight to your e-mail.

Profit and prosper with the best of expert advice - straight to your e-mail.

Sign up

Although ratings can be helpful, they are sometimes manipulated to make businesses look better. Plus, all sorts of biases (both human and statistical) can creep into online rankings.

Inside the ratings game

No one cares much about ratings for a $3 tube of toothpaste. But consumers pay a lot of attention to reviews when it comes to home services, restaurants, hotels and pricey items such as flat-screen TVs and cars. Overall, 54% of Internet shoppers don’t pull out their wallets without first checking consumer reviews, reports Bazaarvoice, a digital marketing firm.

Because ratings are so influential, however, businesses have incentives to inflate their scores. Companies may pepper sites with positive reviews, for instance. They may try to get negative reviews removed, or they may try to bury them so they don’t appear on the site’s first page of comments (which is all many people check). And companies can post negative reviews about competitors.

Sites with lenient review policies seem susceptible to shenanigans. Trip­Advisor, for example, allows people to write reviews without proving that they are legitimate paying customers. Expedia, by contrast, allows reviews only by users who booked travel through the site. A 2014 American Economic Review study found that TripAdvisor included more negative reviews of hotels that had a competitor next door, indicating that hotel owners are more likely to bad-mouth the competition.

Entire businesses have sprung up to help companies scrub their online blemishes. “We dispute bad reviews on behalf of our clients,” says Curtis Boyd, head of one such firm, Future Solutions. If a customer leaves a negative comment that a business wants removed, Boyd may try to get it scrapped for violating a site’s content guidelines, which tend to have some wiggle room. Other strategies include “reasoning” with a customer to remove or upgrade a review, possibly in return for a refund or complimentary service. If all else fails, a company can threaten to sue, claiming a review is defamatory (though it’s a tough case to win unless the write-up is blatantly false).

Review sites, for their part, try to dislodge phony write-ups, which they see as a threat to their credibility (and Web traffic). Yelp deploys algorithms to sift through reviews, filtering write-ups that look overtly promotional, mention a rival business or exhibit other red flags. About 22% of all reviews don’t pass the filters, relegating them to secondary pages.

Angie’s List, a subscription-based review site, goes much further than others, requiring commentators to provide their real names and affirm that their feedback reflects firsthand experiences. The firm also puts itself through an external audit each year to make sure that advertisers aren’t receiving preferential treatment on the site or that ratings aren’t being manipulated.

Amazon, which has been dogged by complaints of promotional reviews, is also trying to crack down on fakes. The company has filed lawsuits against sites that offer sellers a way to buy four- and five-star reviews. It’s also suing more than 1,000 people who allegedly wrote reviews for payments of as little as $5.

Are the reviews real?

Still, what’s fake and what’s real isn’t always clear. In 2012, Britain’s Advertising Standards Authority rebuked TripAdvisor for misleading consumers into believing that all of its reviews were genuine. The ASA told TripAdvisor to stop implying that its reviews were written by “real travelers, or were honest, real or trusted.” (TripAdvisor agreed to stop making such claims on its British site.)

Legally, a consumer review in return for cash or a freebie is considered a paid endorsement, according to the Federal Trade Commission, which requires such disclosures. Yet many companies still provide free or deeply discounted products, such as kitchenware and cosmetics, in return for an “honest” review. Sites such as Honestfew.com broker these types of deals. Moreover, “trusted reviewers” in Amazon’s Vine program can rack up free products in exchange for online feedback. (Amazon labels them as Vine reviews and says they have lower star ratings than other reviews.)

None of this is illegal. But “the law is at best an awkward fit” for such programs, says Eric Robinson, codirector of the Press Law and Democracy Project at Louisiana State University. Indeed, under federal law, sites aren’t liable for blog posts or other claims made by users, giving sites a legal shield against shady review practices.

Furthermore, many sites’ policing software scans primarily for fakes or comments that violate their content guidelines. That leaves room for reviews that may be influenced by a sweetheart deal on a product. If you see reviews of the same product on different sites, you’re not necessarily viewing a wider range of opinions, either. The reviews may be duplicates spread by syndication firms, such as Bazaarvoice, which rounds up consumer comments and sprinkles them around the Web.

On Yelp, a small but vibrant community of consumers who are highly active on the site contribute an outsize number of reviews. Yelp’s “elite squad” writes about half of its restaurant reviews, according to a study by Michael Luca, an assistant professor at Harvard Business School. Although Yelp has amassed more than 100 million reviews, its most influential and “recommended” write-ups are those by elites—on whom Yelp bestows special status, treating them like VIPs.

Being an elite Yelper is like joining an exclusive, invitation-only club. Elites can attend what Yelp calls its “epic parties,” which the firm sponsors with local businesses such as breweries and restaurants. In some cases, elites have received tickets to Broadway-style shows, courtesy of Yelp. “We don’t get financial compensation, but it’s a great perk,” says Eric Rosenberg, 31, an elite Yelper from Portland, Ore., who scored orchestra seats to musicals such as Catch Me If You Can and Jekyll & Hyde.

Elites often rave about Yelp events online and then go back to review the businesses that put them on. Free food and drink may build up positive vibes and prime reviewers to view the providers more favorably when they return and pay out of their own pocket. But for their part, elites generally deny that the parties and freebies influence their opinions. “Businesses can’t suck up to us,” says Yousefi, who is now an elite Yelper and has attended events at, among other places, an organic nail salon, an artisanal pizza place and a Cuban restaurant. “We’re critical people, and you can’t buy our reviews.”

Mike Maslanka, 59, an elite critic in Houston, agrees. After he attended an event for the opening of a Smashburger restaurant, he went back a week later on his own dime, giving it a thumbs-up. “I wrote a good review because I liked it,” he says. “It had nothing to do with the fact that the restaurant comped me the week before.”

How ratings get distorted

Yet even if most people post comments that reflect their experiences, biases inevitably slip in. Many studies have found that online ratings tend to cluster on the positive side. On Amazon and other sites, ratings clump in the upper tiers, according to studies by Paul Pavlou, a data sciences professor at Temple University’s Fox School of Business, and other researchers. That’s also true of Yelp.

A likely explanation is that reviewers tend to visit places they’re predisposed to enjoy and then write favorably about them. Plus, praise for a business or product can be self-reinforcing. Positive reviews hold considerable sway in influencing opinions, according to research by Lev Muchnik, a senior lecturer at the Hebrew University of Jerusalem’s School of Business Administration, and MIT professor Sinan Aral. Negative comments don’t have as much impact in dragging down consumers’ views. And a few, well-placed positive reviews can influence the opinions of subsequent critics, creating misplaced trust, says Aral.

Another issue is that you’re seeing the ratings of a small number of people with strong opinions. If you don’t care about a product, you probably won’t write a review. But you may be inclined to review something you really love or hate. Due to humans’ natural “herd” instincts, positive reviews tend to be more pervasive, contributing to ratings bubbles, says Pavlou.

Finally, there’s the impact of social media. Bombarded with information online, we tend to rely on social media cues—such as article “shares” or “likes”—to form our own judgments, says Muchnik. These signals can trigger herding around a product, and people often amplify the effect by contributing their own likes and shares. What’s more, sites such as Facebook and Twitter use algorithms that interpret social signals and our own online behavior to curate the information that reaches us. It then becomes harder to determine if we’re evaluating a product based on our independent thinking or on signals bopping around in our social media bubble.

When to look at the stars

Ultimately, experts say reviews and ratings are useful. If you’re shopping for a costly product, seeing what other people say can help you make a more informed decision. Tips from other consumers about a hotel or restaurant can be invaluable if you’re on the road. Ratings are also critical if you use “sharing economy” businesses, such as Airbnb and Uber, which feature testimonials from other travelers. Whatever the product, home in on things that matter to you (say, bang for the buck). And don’t gloss over negative comments, which can be more insightful than heaps of praise.

Sure, biases are inevitable. And professional critics aren’t immune to influence, either—they, too, are showered with freebies, from swag at tech conferences and Hollywood junkets to lavish meals and trips. At least with online reviews, you can see the gamut of opinions. And if you’re persistent and click through many comments, you’ll find a diverse array of views. In the end, says Muchnik, “ratings do represent the underlying quality of a product.”