CARE 框架：让共情强化学习中的支持优先级随对话状态演化

View PDF HTML (experimental)

Abstract:We identify a fundamental mismatch in empathetic reinforcement learning: support priorities evolve with the dialogue state, yet existing methods typically optimize predefined reward specifications that remain fixed across turns. To model these evolving support priorities, we organize empathetic support along cognitive, affective, and proactive empathy, and propose Context-Adaptive Rubric Evolution (CARE). At each turn, CARE generates a context-adaptive rubric by adjusting both the weights of these three empathy dimensions and their fine-grained evaluation criteria. The rubric generator is trained with turn-level rubric supervision and human preference data through supervised fine-tuning followed by preference-based reinforcement learning, and then serves as an adaptive reward interface for online empathetic RL. Integrated with both RLVER and MICA, CARE achieves state-of-the-art performance across SentientBench, EQBench3, and EMPA under three independent LLM judges. Notably, on EMPA, CARE improves EPM-Idx over the strongest baseline by at least 13 points under all three judges, including an increase from 28.11 to 83.54 under Gemini-2.5-Pro. Further analyses show that learned rubric priorities systematically vary across dialogue stages and user emotions, demonstrating that CARE adapts what is rewarded as support needs evolve. Code is available at this https URL.

Subjects:	Artificial Intelligence (cs.AI)
Cite as:	arXiv:2609.34249 [cs.AI]
	(or arXiv:2609.34249v3 [cs.AI] for this version)
	https://doi.org/10.48550/arXiv.2609.34249 arXiv-issued DOI via DataCite

Submission history

From: Pengyu Huang [view email]
[v1] Mon, 28 Sep 2026 03:55:21 UTC (3,598 KB)
[v2] Wed, 14 Oct 2026 02:44:40 UTC (3,598 KB)
[v3] Wed, 4 Nov 2026 10:29:50 UTC (4,467 KB)