层级化嵌入主题模型 hETM:在嵌入空间中建模文档与章节层级
Modeling Document and Section Hierarchies in Embedding Space
AI 导读
研究者提出层级化嵌入主题模型 hETM,通过嵌入空间中的层级先验将文档主题分布与其各章节主题分布耦合,并推导出摊销变分推理方法。在法律文书、公司与政府报告、在线论坛等结构化语料上,hETM 的 held-out perplexity 达到或接近平面与层级主题模型基线中的最低值,主题质量相当或更高,且在多数语料的下游聚类任务中取得最佳表现。
正文 · 原文
Abstract:Embedded topic models integrate word embeddings into the topic generation process, producing coherent topics and robust performance on large vocabularies. However, each document is represented by a single topic distribution, ignoring its internal structure. Many real world collections, including legal documents, company and government reports, and online discussion forums, exhibit hierarchical organization in which documents consist of multiple sections that often address distinct subtopics. We introduce the Hierarchical Embedded Topic Model (hETM), a structured topic model that couples the topic distribution of a document and those of its sections through a hierarchical prior in the embedding space. We derive an amortized variational inference procedure for hETM and evaluate it on structured collections from each of these domains. Experimental results show that hETM achieves the lowest or near-lowest held-out perplexity among flat and hierarchical topic model baselines, along with comparable or higher topic quality. It also performs strongly on a downstream clustering task, achieving the best clustering performance on most of the collections.
| Subjects: | Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.21980 [stat.ML] |
| (or arXiv:2611.21980v1 [stat.ML] for this version) | |
| https://doi.org/10.48550/arXiv.2611.21980 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kriste Krstovski [view email]
[v1]
Mon, 16 Nov 2026 23:09:30 UTC (108 KB)
来源:arXiv cs.AI · arxiv.org