Online health care reviews tank following COVID pandemic

After the COVID-19 pandemic struck, online reviews of health care facilities dropped significantly, and they have not yet fully recovered, according to a new analysis led by researchers at the Perelman School of Medicine at the University of Pennsylvania. More than half of reviews on the online platform, Yelp, now are negative, flipping the pre-COVID picture. The findings are published today in JAMA Network Open.

“Online reviews can tell us information about the patient experience that traditional reporting metrics, like hospital-administered patient experience surveys, might miss,” said the study’s lead author, Neil Sehgal, ME, an associate fellow in the Penn Leonard Davis Institute of Health Economics. “These reviews can help hospitals understand what matters most to patients and their support networks in near real time.”

By analyzing all reviews of health care facilities in the United States on the online platform Yelp dated from 2014 through 2023, Sehgal, co-author Anish Agarwal, MD, an assistant professor of Emergency Medicine, and their team saw that the percentage of positive—four- and five-star—reviews dropped from 54.3% before March 2020 (marked as the beginning of the COVID pandemic in the United States) to 47.9% after.

In fact, from the latter half of 2021 on, the researchers found that positive reviews were never more than 50%.

“We analyzed Yelp reviews of health facilities nationwide—which includes hospitals, urgent care centers, doctors’ offices, and more,” said Sehgal. “Our analysis shows how public perception of health care changed after COVID-19. And with this information, hopefully, health care professionals can work to improve upon what’s been driving these opinions.”

Feedback ups and downs

In analyzing the reviews, the researchers used a language processing technique to tease out the most common topics from the reviews, organized them into themes, and measured how they changed over time.

The themes that saw the greatest change in mentions between the pre- and post-COVID periods were “insurance and billing issues” and “customer service and staff behavior.”

One theme mentioned less in negative reviews after the arrival of COVID—and a renewed, supercharged focus on hygiene: Facility cleanliness.

Trends emerge along ethnic and geographic lines

While positive reviews of health care facilities on Yelp, as a whole, declined over time, health care facilities in rural areas already had lower ratings at the start of the time period examined, and the differences became more significant after COVID struck. Post-COVID, rural facilities were 23% less likely to have positive reviews, compared to health facilities in urban areas, which were 7% less likely to have positive reviews.

Looking at data on race, researchers found that health facilities in the sample of neighborhoods that they analyzed that had higher proportions of Black or white residents were more likely to have significant declines in positive reviews.

But when it came to areas with higher-than-average Hispanic populations, facilities still saw a decline in positive ratings, though the dips were not as pronounced.

“At the moment, we cannot understand what is driving these changes among different populations,” Agarwal said. “But this could help us think critically on what questions to ask next and how to better deliver care around the nation in the future.”

Researchers also identified ethnic correlations around the most mentioned complaints. Reviews for facilities in areas with higher Black and/or Hispanic populations criticized “insurance and billing issues” more often, while the facilities in areas with higher white populations found fault with “wait times” more often.

“Many of the themes have clear overlap but are distinct across geographic areas, which is interesting and begins to highlight a growing need for health systems and facilities to be more intentional and tailored to their local communities’ needs—and opinions,” said Agarwal.

Moving forward, the researchers hope to look deeper into the content of the reviews to better understand how attitudes and opinions break down along socio-ethnographic lines, giving health care professionals better tools for discussing issues that are important to individuals and assuaging concerns.