Online rating systems are meant to help consumers make better decisions. But, in reality, people are often overwhelmed with online ratings on everything from doctors to hotels.
One impact of the rating data deluge is the dilution of its value to consumers, argues Rick Watson, University of Georgia Regents Professor and J. Rex Fuqua Distinguished Chair for Internet Strategy at UGA’s Terry College of Business.
Watson, who studies how people and companies interact with data, said the problem arises because the bulk of ratings cluster in the top third of the scale. When every product is rated 9 out of 10 or 4 out of 5, it’s hard for consumers to pick the best or weed out the truly awful options.
This phenomenon, which Watson calls information compression, affects how many markets function and the behavior of service providers such as doctors and restauranteurs. “People generally think more data is better, but this is a case where more data doesn’t improve your ability to make better decisions,” Watson said. “
Watson published his findings in “A Theory of Information Compression: When Judgments are Costly”. The paper was accepted by Information Systems Research and is available at the King’s College London research portal.
Watson worked with fellow UGA management information systems researcher Amrit Tiwana, the L. Edmund Rast Professor of Business at the Terry College, and marketing professors Kirk Plangger of King’s College London and Leyland Pitt of Simon Fraser University in Canada to develop and document the theory.
The phenomenon occurs because of a feedback loop that emerges when service or product providers change their behavior in anticipation of online reviews.
Doctors, for instance, might prescribe antibiotics or pain medications that patients think they need because they fear getting pinged on their online reviews if they don’t meet patients’ expectations.
“Compression occurs when a decision maker acts in anticipation of their rating on a public system, such as Yelp.” Watson said. “Market signals, such as ratings, when compressed into a small range are of little value for consumers because they don’t differentiate among options.”
When rating scores are not useful, consumers’ search costs increase because they must read the reviews. However, these can also be compromised because some reviewers have been sued for expressing a critical opinion, further diminishing the value of online reviews.
“It’s counter to everything people think about databases and data technology,” Watson said. “We’re in this data analytics world and default to, ‘We’ve got all this data; we can make better decisions.’ That’s proving not to be the case in some situations.”
Watson said once they were aware of the phenomenon, the researchers saw examples everywhere — from grade inflation possibly fueled by students ranking their professors to auditors being influenced by the large sums their firm earn from consulting fees.
“If you’re an executive and surround yourself with yes people, that’s information compression,” Watson said. “You can imagine Putin is only hearing from people saying, ‘you’re doing a great a job,’ and that contributes to his actions.”
In some ways, information compression is a phenomenon that was always there, and online rating systems make it more visible.
“I think it’s a problem we’re going to have to live with, but certainly any researchers using these data need to be aware of its possible limitation,” he said.
That said, Watson offered suggestions for mitigating its impacts.
In organizations, leaders might create anonymous channels for suggestions to receive advice and opinions that break from the organization’s party line.
Online rating systems could improve by scrapping simplistic numeric or star-based ratings.
“A single numerical rating is rather subjective,” Watson said. “You could ask four or five questions, like, ‘Was the scheduled appointment on time?’ or ‘Did you feel the physician listened to you?’ That should give consumers better information for decision-making.”