TopicTag: Automatic Annotation of NMF Topic Models Using Chain of Thought and Prompt Tuning with LLMs

Keywords:

nmf, topic labeling, llm, chain of thought, prompt tuning

Citation:

Selma Wanna, Nicholas Solovyev, Ryan Barron, Maksim E. Eren, Manish Bhattarai, Kim Ø. Rasmussen, and Boian S. Alexandrov. 2024. TopicTag: Automatic Annotation of NMF Topic Models Using Chain of Thought and Prompt Tuning with LLMs. In Proceedings of the ACM Symposium on Document Engineering 2024 (DocEng ‘24). Association for Computing Machinery, New York, NY, USA, Article 8, 1–4. https://doi.org/10.1145/3685650.3685667

BibTeX:

@inproceedings{10.1145/3685650.3685667,
author = {Wanna, Selma and Solovyev, Nicholas and Barron, Ryan and Eren, Maksim E. and Bhattarai, Manish and Rasmussen, Kim \O{}. and Alexandrov, Boian S.},
title = {TopicTag: Automatic Annotation of NMF Topic Models Using Chain of Thought and Prompt Tuning with LLMs},
year = {2024},
isbn = {9798400711695},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3685650.3685667},
doi = {10.1145/3685650.3685667},
abstract = {Topic modeling is a technique for organizing and extracting themes from large collections of unstructured text. Non-negative matrix factorization (NMF) is a common unsupervised approach that decomposes a term frequency-inverse document frequency (TF-IDF) matrix to uncover latent topics and segment the dataset accordingly. While useful for highlighting patterns and clustering documents, NMF does not provide explicit topic labels, necessitating subject matter experts (SMEs) to assign labels manually. We present a methodology for automating topic labeling in documents clustered via NMF with automatic model determination (NMFk). By leveraging the output of NMFk and employing prompt engineering, we utilize large language models (LLMs) to generate accurate topic labels. Our case study on over 34,000 scientific abstracts on Knowledge Graphs demonstrates the effectiveness of our method in enhancing knowledge management and document organization.},
booktitle = {Proceedings of the ACM Symposium on Document Engineering 2024},
articleno = {8},
numpages = {4},
keywords = {chain of thought, llm, nmf, prompt tuning, topic labeling},
location = {San Jose, CA, USA},
series = {DocEng '24}
}