AI tools can make education materials more patient friendly

Artificial intelligence (AI) tools significantly improve the readability of online patient education materials (PEMs), making them more accessible, a new study shows.

Led by researchers at NYU Langone Health, the study focused on the readability of PEMs available on the websites of the American Heart Association (AHA), American Cancer Society (ACS), and American Stroke Association (ASA). According to the researchers, these materials help patients make decisions about their health care but often exceed the recommended reading level of grade 6, making them difficult for many patients to understand.

For the study, researchers evaluated the capabilities of three large language models (LLMs)—ChatGPT, Gemini, and Claude—to optimize the readability of PEMs without compromising accuracy. These generative AI tools are designed to simplify complex texts by predicting the next word in a sentence based on extensive Internet data. This next-word prediction gives such models the ability to rewrite any article in simpler language as directed.

Published online April 10 in the Journal of Medical Internet Research, the study involved 60 randomly selected PEMs from the AHA, ACS, and ASA websites. Researchers prompted the LLMs to simplify the reading level of the materials. Results showed that the original readability scores were significantly above the recommended level of grade 6, with mean grade-level scores of 10.7, 10, and 9.6, respectively.

After optimization by the LLMs, readability scores improved significantly across all three websites. ChatGPT improved readability to a mean grade level of 7.6, Gemini to 6.6, and Claude to 5.6. Word counts were also significantly reduced, making the materials more concise.

“Our study shows that widely used large language models have the potential to transform patient education materials into more readable content, which is essential for patient empowerment and better health outcomes,” said study senior author Jonah Feldman, MD, medical director of transformation and informatics at NYU Langone.

“Our findings demonstrate that even expert-composed education materials, which are already patient-directed, can benefit from AI-driven improvements,” said Feldman, who also serves as an assistant professor at NYU Grossman Long Island School of Medicine.

This study, the researchers say, provides an example of how healthcare organizations can apply AI to make clinical communication more patient friendly. Prior studies demonstrated the capabilities of AI models to create patient-focused explanations of heart test results, to draft responses to electronic advice queries, and to generate human-friendly summaries of complex medical reports.

“The breadth of possible AI offerings shows how technology can be leveraged to transform the patient experience across health care systems, and not just in the United States,” said study co-author Paul Testa, MD, JD, MPH, chief health informatics officer at NYU Langone.

“These studies are not just theoretical—after demonstrating their effectiveness, we are actively putting these AI tools into practice,” said Testa, who is also a clinical professor at NYU Grossman School of Medicine.

According to Testa, the NYU Langone team is already using the same AI tools in a randomized controlled trial that incorporates AI-generated, patient-friendly summaries for hospital discharge instructions, with the goal to evaluate their effectiveness in improving patient comprehension and satisfaction. The researchers hope to show that providing clear and accessible discharge instructions will help ensure better postdischarge care and smoother transitions.

“Generating real-world evidence through randomized trials is crucial for validating the effectiveness of AI tools in clinical settings,” said study co-author Jonah Zaretsky, MD, associate chief of medicine at NYU Langone Hospital—Brooklyn. “This approach ensures that the AI-generated documentation is not only accurate but also genuinely beneficial for patients and their families,” added Zaretsky, a clinical assistant professor at NYU Grossman School of Medicine.

Besides Feldman, Testa, and Zaretsky, NYU Langone researchers involved in the study were lead author John Will, and co-authors Mahin Gupta and Aliesha Dowlath.

Artificial intelligence (AI) tools significantly improve the readability of online patient education materials (PEMs), making them more accessible, a new study shows.

Led by researchers at NYU Langone Health, the study focused on the readability of PEMs available on the websites of the American Heart Association (AHA), American Cancer Society (ACS), and American Stroke Association (ASA). According to the researchers, these materials help patients make decisions about their health care but often exceed the recommended reading level of grade 6, making them difficult for many patients to understand.

For the study, researchers evaluated the capabilities of three large language models (LLMs)—ChatGPT, Gemini, and Claude—to optimize the readability of PEMs without compromising accuracy. These generative AI tools are designed to simplify complex texts by predicting the next word in a sentence based on extensive Internet data. This next-word prediction gives such models the ability to rewrite any article in simpler language as directed.

Published online April 10 in the Journal of Medical Internet Research, the study involved 60 randomly selected PEMs from the AHA, ACS, and ASA websites. Researchers prompted the LLMs to simplify the reading level of the materials. Results showed that the original readability scores were significantly above the recommended level of grade 6, with mean grade-level scores of 10.7, 10, and 9.6, respectively.

After optimization by the LLMs, readability scores improved significantly across all three websites. ChatGPT improved readability to a mean grade level of 7.6, Gemini to 6.6, and Claude to 5.6. Word counts were also significantly reduced, making the materials more concise.

“Our study shows that widely used large language models have the potential to transform patient education materials into more readable content, which is essential for patient empowerment and better health outcomes,” said study senior author Jonah Feldman, MD, medical director of transformation and informatics at NYU Langone.

“Our findings demonstrate that even expert-composed education materials, which are already patient-directed, can benefit from AI-driven improvements,” said Feldman, who also serves as an assistant professor at NYU Grossman Long Island School of Medicine.

This study, the researchers say, provides an example of how healthcare organizations can apply AI to make clinical communication more patient friendly. Prior studies demonstrated the capabilities of AI models to create patient-focused explanations of heart test results, to draft responses to electronic advice queries, and to generate human-friendly summaries of complex medical reports.

“The breadth of possible AI offerings shows how technology can be leveraged to transform the patient experience across health care systems, and not just in the United States,” said study co-author Paul Testa, MD, JD, MPH, chief health informatics officer at NYU Langone.

“These studies are not just theoretical—after demonstrating their effectiveness, we are actively putting these AI tools into practice,” said Testa, who is also a clinical professor at NYU Grossman School of Medicine.

According to Testa, the NYU Langone team is already using the same AI tools in a randomized controlled trial that incorporates AI-generated, patient-friendly summaries for hospital discharge instructions, with the goal to evaluate their effectiveness in improving patient comprehension and satisfaction. The researchers hope to show that providing clear and accessible discharge instructions will help ensure better postdischarge care and smoother transitions.

“Generating real-world evidence through randomized trials is crucial for validating the effectiveness of AI tools in clinical settings,” said study co-author Jonah Zaretsky, MD, associate chief of medicine at NYU Langone Hospital—Brooklyn. “This approach ensures that the AI-generated documentation is not only accurate but also genuinely beneficial for patients and their families,” added Zaretsky, a clinical assistant professor at NYU Grossman School of Medicine.

Besides Feldman, Testa, and Zaretsky, NYU Langone researchers involved in the study were lead author John Will, and co-authors Mahin Gupta and Aliesha Dowlath.