- Moderator: Usha Lee McFarling, director, Knight Science Journalism Program at MIT; board member, AHCJ
- Tiffani J. Bright, Ph.D., FACMI, assistant professor, Cedars-Sinai Medical Center Department of Computational Biomedicine; co-director, Center for Artificial Intelligence Research and Education
While artificial intelligence holds medical promise, unchecked algorithms threaten patient safety and health equity, said Tiffani Bright, Ph.D., an artificial intelligence researcher, during AHCJ’s fall summit in Los Angeles.
“It doesn’t take a super fancy algorithm to do harm,” said Bright, an assistant professor and co-director of the Center for Artificial Intelligence Research and Education at Cedars-Sinai Medical Center.
“It just takes one basic algorithm that hasn’t been tested to cause a human life harm,” she said.
In the past, health institutions maintained local control over their algorithms to detect and root out bias. “Today, proprietary commercial vendors ship ‘black box’ tools where health systems cannot inspect the underlying logic or training cohorts,” she added.
According to Bright, bias enters health algorithms through three main pathways: biased historical data, asking the wrong clinical questions and failing to act on system alerts.
For instance, algorithms trained primarily on male patient data can deliver inaccurate or detrimental results when used on women. Similarly, models that rely on health care costs to identify sick patients often overlook lower-income individuals who receive less care overall.
Bright said that a lack of adequate independent oversight compounds the problem. She said that most AI tools do not undergo rigorous clinical trials by outside evaluators the way prescription drugs do before they reach patients.
Bright added that an AI system claiming high overall accuracy can still fail specific subgroups, such as Black or Hispanic women, if developers do not evaluate performance across intersectional lines.
While some in the health care field predict AI could automate routine medical visits, Bright said she worries automated care could be disproportionately funneled to disadvantaged groups, making direct human care a luxury and creating a two-tiered health care system.
Bright advised reporters to scrutinize medical AI claims by investigating funding sources, verifying whether tools are tested on diverse populations, and asking whether patients gave informed consent to tools that record medical visits and access patient records.
“Bias is not a bug. It’s a set of choices,” she said, adding that there needs to be greater representation and accountability in how medical technology is built and deployed.
The full recordings of this panel discussion and more from AHCJ’s fall summit 2026 are available on our website.
