arXiv cs.AI· Peter Romero, Haotian Li, Hugo Save, Luning Sun, Zachary R. Tidler, Kazuya Horibe, Joe Watson, Daniel Romero-Alvarado, Israel Mason-Williams, Fernando Mart\'inez-Plumed, Jos\'e Hern\'andez-Orallo·· 2 小时前AI 评分35
28 项 AI 倾向量表发布：评分手册、合成题库与标注
A Battery for AI Propensity Measurement: Rubrics, Synthetic Item Banks, and Annotations
AI 导读
Peter Romero 等人在 arXiv 发布一套包含 28 项 AI 倾向的测量量表，配有专家撰写的评分手册、合成数据集和标注好的需求区间，基于 Romero-Alvarado 等人（2026）的方法构建。这些倾向源自心理测量学、心理学与行为研究，便于将 AI 实验结果与人类行为研究的既有构念对照。评分手册可用于标注现有基准，合成数据集各自仅对单一倾向敏感，可快速生成倾向画像。
正文 · 原文
Authors:Peter Romero, Haotian Li, Hugo Save, Luning Sun, Zachary R. Tidler, Kazuya Horibe, Joe Watson, Daniel Romero-Alvarado, Israel Mason-Williams, Fernando Martínez-Plumed, José Hernández-Orallo
Abstract:We release a battery of 28 propensities for AI research, with expert-written rubrics, synthetic datasets, and annotated demand intervals, built with the method of Romero-Alvarado et al. (2026). We selected a wide collection of generic propensities that may influence the behavior of AI systems and humans. They are inspired by psychometric, psychological, and behavioral research, so that experimental results can be understood in the context of familiar constructs previously used also for human behavior research. The rubrics are useful for annotating existing benchmarks whose items may be relevant or not for one or more propensities. For easier profiling, however, the synthetic datasets are defined in such a way that they should be sensitive to one propensity only, allowing for a quick prototyping of propensity profiles using the battery of datasets. We also outline directions for future propensity research.
| Comments: | 10 pages, 2 figures, instrument release |
| Subjects: | Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Machine Learning (cs.LG) |
| ACM classes: | I.2.0; J.4 |
| Cite as: | arXiv:2610.34614 [cs.AI] |
| (or arXiv:2610.34614v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34614 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Peter Romero PhD [view email]
[v1]
Sun, 25 Oct 2026 22:22:11 UTC (215 KB)
来源：arXiv cs.AI · arxiv.org