IFAAB 数据集：审计 LLM 生成的印度金融建议中的社会经济与人口偏差
The Indian Financial Automated Advice Bias (IFAAB) Dataset : Auditing Socioeconomic and Demographic Disparities in LLM-Generated Financial Advice in India
AI 导读
研究者发布 IFAAB 基准，包含 31,734 条 GPT-4o 为不同印度虚构人设生成的金融建议响应，用于反事实审计性别、种姓、教育、居住地等维度的差异。审计显示男性标签档案获得的初始总资源高出 60.1%、建议后总资源高出 46.8%；加入地区信息后该差距降至 7.9%，显示强地理条件性。女性更常被导向谨慎储蓄和基础理财教育，男性则更多获得投资与创业类建议。
正文 · 原文
Abstract:Large language models are increasingly used for financial guidance, yet little is known about whether their advice changes across social identities in Global South contexts. We introduce the Indian Financial Automated Advice Bias (IFAAB) benchmark, a counterfactual audit dataset of 31,734 GPT-4o-generated financial-advice responses for synthetic Indian personas varying by gender, caste, education, employment, residence, age, and district. Holding the underlying request fixed, we evaluate disparities in LLM-implied wealth, income, savings, projected gains, textual similarity, advice topics, and qualitative bias judgments. In the gender-only audit, male-labelled profiles receive 60.1\% higher initial total resources and 46.8\% higher post-advice total resources than comparable female-labelled profiles; in the caste-augmented audit, the post-advice total-resource gap remains 49.4\%. Adding district information reduces the aggregate post-advice total-resource gender gap to 7.9\%, revealing strong geographic conditioning. Standalone caste comparisons also show systematic stratification: relative to General-category profiles, OBC, SC, and ST profiles receive significantly lower post-advice total resources, respectively. Textual audits show low semantic similarity between paired gendered responses, with an average TF--IDF cosine similarity score of 0.44. Qualitative analysis shows that women are more often steered toward cautious saving, community support, and basic financial literacy, while men receive more growth-oriented advice involving investment, entrepreneurship, and asset accumulation. IFAAB provides a reusable benchmark for evaluating fairness, suitability, and consumer-protection risks in LLM-mediated financial advice, and highlights the need for locally grounded, intersectional evaluation of AI systems in high-stakes economic domains.
| Subjects: | Computers and Society (cs.CY); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.38239 [cs.CY] |
| (or arXiv:2610.38239v1 [cs.CY] for this version) | |
| https://doi.org/10.48550/arXiv.2610.38239 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kriti Mahajan [view email]
[v1]
Wed, 28 Oct 2026 03:39:59 UTC (13,740 KB)
来源：arXiv cs.AI · arxiv.org