Nearly five years ago, Emory University radiologist Judy Gichoya, M.D., spotted a call for research papers on health disparities. She and her colleagues set out to train a machine learning algorithm using a diverse set of medical images to show that using more inclusive data could lead to more precise results for everyone — and thus, increased health equity. “We were hoping to say that if you have more diversity in the dataset, you’ll mitigate these biases,” Gichoya said.
To the researchers’ surprise, the AI-based tool learned how to predict a person’s race from radiological images such as X-rays, mammograms, and chest CTs. Precisely why this occurred and how the algorithm gleaned people’s racial information is still unclear — differences between racial groups are not based on biology. Instead of showing that diverse training data would be sufficient to improve health outcomes, “we landed on this really fascinating question,” Gichoya said.
As more AI-based tools are implemented across various aspects of medical education and health, journalists can follow these questions to find timely stories on how AI-based algorithms are changing clinical training and patient care. Here are three story angles to consider.
Local stories
Just as schools follow specific curricula, AI tools are trained to do their jobs using certain data. In a 2021 study, researchers found that an algorithm to read CT scans developed at one institution was up to 10% less accurate when used to diagnose data from patients at two other medical centers. The inaccuracies were also greater amongst scans from Black patients.
Journalists can seek out stories about new AI-based medical imaging tools being adopted by local hospitals or imaging facilities. They can also explore where and how technologies were developed, and ask experts how differences in data demographics may lead to health disparities and what safeguards can be implemented.
Critical care
Algorithms to predict the severity of a person’s illness or decide on a course of treatment are widely used in ICUs. Calculators such as the MELD score, SOFA score, or others are routinely used to gauge a person’s risk of liver failure, sepsis, or other critical illnesses. These tools — and the clinicians who use them — must factor in many sources of information, ranging from continually recorded vital signs to imaging results and other tests to make decisions. “It’s very labor and resource-intensive,” Gichoya said, making it an opportune space to implement AI-based alternatives.
But developing accurate predictive tools is difficult, she adds, because AI tools may pick up on patterns in health care that are invisible to the human eye. “There are patterns that exist in health care, whether we humans acknowledge them or not, that are the products of systemic racism,” she said.
Journalists can find story ideas by learning how local hospitals are incorporating algorithms in ICUs and tracking differences in outcomes due to AI tools.
Medical training
Researchers are starting to explore using LLMs such as ChatGPT for medical education, creating case studies for learning, or interactive simulations for trainees to practice their diagnostic skills.
In a recent study, Gichoya and her colleagues found that GPT-4 repeated common biases, particularly if a disease was more prevalent in certain racial or ethnic groups than others. For example, the chronic immune condition sarcoidosis is more frequently seen among Black people and women than other racial groups or men. In their study, the researchers found that the model generated an example of a Black patient 966 out of 1000 times, a female patient 835 times, and a Black woman 810 times.
“It’s not surprising that the model is just going to reflect what society reflects,” Gichoya said. However, this over-representation in AI-generated examples could lead to clinicians over-diagnosing the condition amongst Black women and underestimating risks in other groups.
Journalists should investigate how AI is used in medical training and seek out examples where researchers are solving the problem of algorithmic biases.
Resources
- AI image generators often give racist and sexist results: can they be fixed? — Nature.
- Rooting Out AI’s Biases — Hopkins Bloomberg Public Health magazine.
- Epic’s overhaul of a flawed algorithm shows why AI oversight is a life-or-death issue — STAT.
- AI in medicine needs to be carefully deployed to counter bias – and not entrench it — NPR.
