Exploring the role of AI in palliative care and decision-making

A few years ago, when second-year School of Nursing Ph.D. student Oonjee Oh was working as a nurse on a general surgery unit in South Korea, she was struck by the feeling that many patients weren’t getting the opportunity to think about end-of-life care options.

“There were a lot of seriously ill patients who were going to the ICU, back to our unit, and then, a few days later, they would go back again to the ICU and then back to our unit.” Oh says, “And even though these patients were very seriously ill, no one was really talking about comfort-focused care, or whether these aggressive types of treatments were really in their best interest.”

The observation put her on a path to her current research focus on palliative care—first to how families decide to enroll a loved one with dementia into hospice, and then to an interest in the role of artificial intelligence in end-of-life decision-making and caregiver support.

Oh was the lead author on the paper “The ethical dimensions of utilizing Artificial Intelligence in palliative care,” which was published in the journal Nursing Ethics in November. This paper grew out of work she did in Inquiry and Nursing, a course she took her first semester at Penn.

The paper applies the moral principles of beneficence, nonmaleficence, autonomy, justice, and explicability to examine the ethical dimensions of three hypothetical use cases: machine learning algorithms that predict patient mortality, natural language processing models that capture the signals of psychological distress from clinical notes, and chatbots that provide informational and emotional support to caregivers.

“As technology advances, ethical principles and practices frequently lag behind, and AI is no exception,” says faculty advisor and co-author Connie M. Ulrich, the Lillian S. Brunner Chair in Medical and Surgical Nursing, professor of nursing, and professor of medical ethics and health policy.

She says Oh’s paper “encourages us to consider both the benefits and challenges of using AI in supportive care for patients and families. These include the importance of autonomy and personal decision-making, the possibility of distress from using AI, biases in algorithms, and treatment predictions, among other concerns.”

Oh’s other faculty advisor and co-author is Penn Integrates Knowledge University Professor George Demiris, who has joint appointments in the School of Nursing and the Perelman School of Medicine. It was the opportunity to work with faculty across disciplines that made Penn Nursing appealing, Oh says.

“I always wanted to learn more about how technology could play a role in this space, and Dr. Demiris leads various projects on telehealth and smart homes for older adults or patients in hospice,” Oh says.

“In addition to how technology can support end-of-life care, I have always been interested in how patients and families make the decision for palliative care and the conflicts that arise during that process—ethical dilemmas are often intertwined with those issues—so I find Dr. Ulrich’s expertise in bioethics very inspirational and insightful.”

Oh says Ulrich’s work inspired her to pursue a dual degree with a Master of Bioethics.

Oh’s interest in data science and AI, she says, started while she was pursuing her master’s degree in nursing in Seoul at the peak of the COVID pandemic. Unable to go out into the field to interview or collect data from patients, she focused her research on national survey data and became interested in analyzing big data. Her interest in coding and advanced statistical models led her to learn more about various research methodologies at a doctoral level.

When she started her Ph.D. in the fall of 2023, ChatGPT “was starting to get a lot of attention as a potential tool to advance health care and its workflow,” she says. “Since it is so new, people were excited, but they were also very concerned about how these generative AI tools could be used to help patients and families, especially in the end-of-life space, since a lot of vulnerable populations would be impacted.”

In December 2023, Oh attended “ChatGPT and Aging: Implications of Generative AI for Gerontology,” a two-day roundtable discussion that Penn Nursing held with the Penn Artificial Intelligence and Technologies Collaboratory for Healthy Aging (PennAITech), of which Demiris is a principal investigator. Oh is also an Innovation Fellow with PennAITech, which involves getting invited to conferences and helping evaluate applications for pilot grants.

Oh worked with Hannah Cho—another Penn Nursing Ph.D. student and Innovation Fellow who researches palliative care—on a literature review of how artificial intelligence is being used for older adults. Publishing their findings this month in The Journals of Gerontology, Series A: Biological Sciences and Medical Sciences, they found that AI system design processes often overlook the needs of older adults.

Oh, Cho, and Sang Bin You, also a Penn Nursing Ph.D. student and a PennAITech Innovation Fellow, were the student members of the planning committee for the two-day AI and Nursing Science workshop that Penn Nursing held in January. Demiris chaired the committee.

“Nurses have to inform the design and development of AI tools for health care,” Demiris says. “Oonjee’s work advances our insights into the role AI can play in nursing science and practice with an emphasis on systems that promote fairness and accountability and capture the patient’s voice.”

A few years ago, when second-year School of Nursing Ph.D. student Oonjee Oh was working as a nurse on a general surgery unit in South Korea, she was struck by the feeling that many patients weren’t getting the opportunity to think about end-of-life care options.

“There were a lot of seriously ill patients who were going to the ICU, back to our unit, and then, a few days later, they would go back again to the ICU and then back to our unit.” Oh says, “And even though these patients were very seriously ill, no one was really talking about comfort-focused care, or whether these aggressive types of treatments were really in their best interest.”

The observation put her on a path to her current research focus on palliative care—first to how families decide to enroll a loved one with dementia into hospice, and then to an interest in the role of artificial intelligence in end-of-life decision-making and caregiver support.

Oh was the lead author on the paper “The ethical dimensions of utilizing Artificial Intelligence in palliative care,” which was published in the journal Nursing Ethics in November. This paper grew out of work she did in Inquiry and Nursing, a course she took her first semester at Penn.

The paper applies the moral principles of beneficence, nonmaleficence, autonomy, justice, and explicability to examine the ethical dimensions of three hypothetical use cases: machine learning algorithms that predict patient mortality, natural language processing models that capture the signals of psychological distress from clinical notes, and chatbots that provide informational and emotional support to caregivers.

“As technology advances, ethical principles and practices frequently lag behind, and AI is no exception,” says faculty advisor and co-author Connie M. Ulrich, the Lillian S. Brunner Chair in Medical and Surgical Nursing, professor of nursing, and professor of medical ethics and health policy.

She says Oh’s paper “encourages us to consider both the benefits and challenges of using AI in supportive care for patients and families. These include the importance of autonomy and personal decision-making, the possibility of distress from using AI, biases in algorithms, and treatment predictions, among other concerns.”

Oh’s other faculty advisor and co-author is Penn Integrates Knowledge University Professor George Demiris, who has joint appointments in the School of Nursing and the Perelman School of Medicine. It was the opportunity to work with faculty across disciplines that made Penn Nursing appealing, Oh says.

“I always wanted to learn more about how technology could play a role in this space, and Dr. Demiris leads various projects on telehealth and smart homes for older adults or patients in hospice,” Oh says.

“In addition to how technology can support end-of-life care, I have always been interested in how patients and families make the decision for palliative care and the conflicts that arise during that process—ethical dilemmas are often intertwined with those issues—so I find Dr. Ulrich’s expertise in bioethics very inspirational and insightful.”

Oh says Ulrich’s work inspired her to pursue a dual degree with a Master of Bioethics.

Oh’s interest in data science and AI, she says, started while she was pursuing her master’s degree in nursing in Seoul at the peak of the COVID pandemic. Unable to go out into the field to interview or collect data from patients, she focused her research on national survey data and became interested in analyzing big data. Her interest in coding and advanced statistical models led her to learn more about various research methodologies at a doctoral level.

When she started her Ph.D. in the fall of 2023, ChatGPT “was starting to get a lot of attention as a potential tool to advance health care and its workflow,” she says. “Since it is so new, people were excited, but they were also very concerned about how these generative AI tools could be used to help patients and families, especially in the end-of-life space, since a lot of vulnerable populations would be impacted.”

In December 2023, Oh attended “ChatGPT and Aging: Implications of Generative AI for Gerontology,” a two-day roundtable discussion that Penn Nursing held with the Penn Artificial Intelligence and Technologies Collaboratory for Healthy Aging (PennAITech), of which Demiris is a principal investigator. Oh is also an Innovation Fellow with PennAITech, which involves getting invited to conferences and helping evaluate applications for pilot grants.

Oh worked with Hannah Cho—another Penn Nursing Ph.D. student and Innovation Fellow who researches palliative care—on a literature review of how artificial intelligence is being used for older adults. Publishing their findings this month in The Journals of Gerontology, Series A: Biological Sciences and Medical Sciences, they found that AI system design processes often overlook the needs of older adults.

Oh, Cho, and Sang Bin You, also a Penn Nursing Ph.D. student and a PennAITech Innovation Fellow, were the student members of the planning committee for the two-day AI and Nursing Science workshop that Penn Nursing held in January. Demiris chaired the committee.

“Nurses have to inform the design and development of AI tools for health care,” Demiris says. “Oonjee’s work advances our insights into the role AI can play in nursing science and practice with an emphasis on systems that promote fairness and accountability and capture the patient’s voice.”