在奖励不可观测下激励完全信息智能体的学习策略研究
Learning to Incentivize Perfect-Knowledge Agents with Unobserved Rewards
AI 导读
这篇 arXiv 论文研究多臂老虎机框架下的重复委托-代理博弈：委托人为每个臂提供激励，代理选择臂最大化自身收益，但委托人无法观测代理的奖励。论文利用激励序列与所选臂序列构建代理期望奖励的估计器（类比多臂老虎机中的在线逆优化），并针对代理完全了解自身各臂期望奖励的情形设计了策略，证明其达到对数因子内的平方根遗憾上界，最后以他汀类药物长期治疗依从性个性化激励设计的数值模拟展示框架适用性。
正文 · 原文
Abstract:Motivated by a number of real-world applications from domains like healthcare and sustainable transportation, in this paper we study a scenario of repeated principal-agent games within a multi-armed bandit (MAB) framework, where: the principal gives a different incentive for each bandit arm, the agent picks a bandit arm to maximize its own expected reward plus incentive, and the principal observes which arm is chosen and receives a reward (different than that of the agent) for the chosen arm. Designing policies for the principal is challenging because the principal cannot directly observe the reward that the agent receives for their chosen actions, and so the principal cannot directly learn the expected reward using existing estimation techniques. As a result, the problem of designing policies for this scenario, as well as similar ones, remains mostly unexplored. In this paper, we construct a policy that achieves a low regret (i.e., square-root regret up to a log factor) in this scenario for the case where the agent has perfect-knowledge about its own expected rewards for each bandit arm. We design our policy by first constructing an estimator for the agent's expected reward for each bandit arm. Since our estimator uses as data the sequence of incentives offered and subsequently chosen arms, the principal's estimation can be regarded as an analogy of online inverse optimization in MAB's. Next, we construct a policy that we prove achieves a low regret by deriving finite-sample concentration bounds for our estimator. We conclude with numerical simulations demonstrating the applicability of our framework to personalized adherence-improving incentive design for long-term statin treatments.
| Comments: | 61 pages, 3 figures |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Science and Game Theory (cs.GT); Machine Learning (stat.ML) |
| Cite as: | arXiv:2304.07407 [cs.LG] |
| (or arXiv:2304.07407v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2304.07407 arXiv-issued DOI via DataCite |
Submission history
From: Ilgin Dogan [view email]
[v1]
Fri, 14 Apr 2023 21:57:16 UTC (299 KB)
[v2]
Sun, 7 May 2023 19:30:01 UTC (300 KB)
[v3]
Wed, 4 Nov 2026 08:24:56 UTC (263 KB)
来源：arXiv cs.AI · arxiv.org