Medical AI may look less biased on paper but not in practice, new study finds

Large language models (LLMs) are only as good as the data they learn from.

If their training data contains social biases, the models may unintentionally repeat those biases in their responses.

As their use increases with the rise of generative AI, it has become evident that they contain biases that reinforce stereotypes based on factors like ethnicity, gender or geography.

This is concerning because LLM-based chatbots have become an integral part of the health care system, where health-related stigma can negatively affect one's diagnosis and care. ...