How AI could help triage emergency department care

Anyone who has been to an emergency department for a non-life-threatening event knows the pain of waiting for hours before a bed becomes available for hospital admission for further care. Now, two studies demonstrate that using GPT-4, a large language model created by OpenAI, holds potential for helping emergency department personnel determine which patients need the most urgent treatment and which patients ultimately will require admission to the hospital.

Journalists could find interesting stories by following these and other studies of AI technology in the emergency department setting and by interviewing emergency medicine experts trialing such technologies.

In the first study, published May 7 in JAMA Network Open, researchers at the University of California, San Francisco, input 10,000 pairs of patient information data from recent emergency department visits (minus patient names or other identifying information) into GPT-4 to see if the AI tool could identify which patient had the more severe condition.

The pairs featured one patient with a serious condition such as a stroke and another with a less urgent need such as a broken wrist. AI correctly selected the patient with the more serious condition in 89% of cases. A subgroup of 500 pairs of patient information was then evaluated both by GPT-4 and by physicians. The result? GPT-4 was accurate 88% of the time, a slight edge over the physicians at 86%.

Having AI assist in the triage process could help physicians allocate their time efficiently and serve as a backup for decision-making, study author Christopher Williams, M.D., said in a UCSF story about the study.

“Imagine two patients who need to be transported to the hospital but there is only one ambulance,” he said in the article, “or a physician on call and there are three people paging her at the same time and she has to determine who to respond to first.”

Still, Williams noted that it is not quite ready to use responsibly in an emergency department setting without further validation and clinical trials alongside efforts to erase racial and gender biases.

In a second study, published May 21 in the Journal of the American Medical Informatics Association, investigators at the Icahn School of Medicine at Mount Sinai found that GPT-4 also holds potential for predicting which emergency department patients would be admitted to the hospital.

The researchers input data including triage notes from more than 864,000 patient emergency department visits at seven Mount Sinai hospitals; of these, 159,857 (18.5%) patients were admitted to the hospital. The visits were for a wide variety of clinical conditions. On its first try, the program was 77.5% accurate in predicting admissions; that accuracy improved to 83% when the program was given additional data to learn from.

Doctors and hospital staff theoretically could use the technology to shorten patient wait time and more quickly determine how many beds are needed in a hospital, which emergency department patients should be transferred to inpatient floors and which should be discharged, The Hill’s Alejandra O’Connell-Domenech reported in May.

Still, researchers concluded that while the technology holds promise, it requires enhancements.

“Moving forward,” the study authors wrote, “careful consideration must be given to the design of these systems to ensure they augment rather than complicate the decision-making process.”

Even if such technology is implemented in emergency rooms, doctors will always need to perform their own independent evaluation to determine a patient’s treatment, Ajeet Singh, M.D., a hospitalist and informaticist at Rush University Medical Center, commented to The Hill. These programs do not understand the words they are trained on but merely mimic reasoning by predicting relationships of words to one another, he said.