收听本期播客
阅读正文
Artificial Intelligence (AI) tools are becoming increasingly common in public services across England, but a recent study by the London School of Economics and Political Science has raised important questions about their fairness. More than half of local councils in the country are using these tools to assist overworked social workers by summarizing case notes. While the technology aims to save time, the research suggests it may be creating unequal treatment in how men’s and women’s health needs are assessed.
The study examined real case notes from over 600 adult social care users. Researchers altered only the gender in these notes and analyzed thousands of summaries generated by various AI models. The findings were concerning. For example, when summarizing the condition of an 84-year-old man, an AI tool might highlight a complex medical history and limited mobility. However, for a woman with identical health issues, the same tool could describe her as independent and capable of managing daily tasks. Words such as ‘disabled’ or ‘unable’ appeared far more frequently in summaries about men than women, even when their needs were the same. This discrepancy could result in women receiving less support if care decisions rely on these summaries.
Dr. Sam Rickman, the lead researcher, emphasized that such differences in language might lead to unequal care, particularly for women. He expressed concern that many councils do not disclose which AI models they use or how often they rely on them. Interestingly, not all AI tools showed this bias. The study found that Llama 3, developed by Meta, produced summaries without noticeable gender differences, unlike Google’s Gemma model, which often downplayed women’s health issues.
This issue reflects a wider problem with fairness in technology. AI systems often learn from human language, which can already contain biases, and these biases may then influence the tools’ outputs. As AI becomes more integrated into public services, there is a growing demand for strict testing and transparency to ensure that such technology does not undermine equality. The challenge remains: how can society benefit from AI’s efficiency while preventing unfair treatment? This question is becoming more urgent as reliance on these tools continues to grow.