Publication Details
Learning Document Embeddings Along With Their Uncertainties
Plchot Oldřich, Ing., Ph.D. (DCGM)
Burget Lukáš, doc. Ing., Ph.D. (DCGM)
Gangashetty Suryakanth V
Bayesian methods, embeddings, topic identification.
Majority of the text modeling techniques yield only point-estimates of document
embeddings and lack in capturing the uncertainty of the estimates. These
uncertainties give a notion of how well the embeddings represent a document. We
present Bayesian subspace multinomial model (Bayesian SMM), a generative
log-linear model that learns to represent documents in the form of Gaussian
distributions, thereby encoding the uncertainty in its covariance. Additionally,
in the proposed Bayesian SMM, we address a commonly encountered problem of
intractability that appears during variational inference in mixed-logit models.
We also present a generative Gaussian linear classifier for topic identification
that exploits the uncertainty in document embeddings. Our intrinsic evaluation
using perplexity measure shows that the proposed Bayesian SMM fits the unseen
test data better as compared to the state-of-the-art neural variational document
model on (Fisher) speech and (20Newsgroups) text corpora. Our topic
identification experiments showthat the proposed systems are robust to
over-fitting on unseen test data. The topic ID results show that the
proposedmodel outperforms state-of-the-art unsupervised topic models and achieve
comparable results to the state-of-the-art fully supervised discriminative
models.
@article{BUT168164,
author="Santosh {Kesiraju} and Oldřich {Plchot} and Lukáš {Burget} and Suryakanth V {Gangashetty}",
title="Learning Document Embeddings Along With Their Uncertainties",
journal="IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH AND LANGUAGE PROCESSING",
year="2020",
volume="2020",
number="28",
pages="2319--2332",
doi="10.1109/TASLP.2020.3012062",
issn="2329-9290",
url="https://ieeexplore.ieee.org/document/9149686"
}