Publication Details

Approximate inference: A sampling based modeling technique to capture complex dependencies in a language model

DEORAS, A.; MIKOLOV, T.; KOMBRINK, S.; CHURCH, K. Approximate inference: A sampling based modeling technique to capture complex dependencies in a language model. Speech Communication, 2012, vol. 2012, no. 8, p. 1-16. ISSN: 0167-6393.
Czech title
Přibližná inference: podchycení složitých vztahů v jazykovém modelu pomocí techniky založené na vzorkování.
Type
journal article
Language
English
Authors
Deoras Anoop
Mikolov Tomáš, Ing., Ph.D.
Kombrink Stefan, Dipl.-Linguist.
Church Kenneth
URL
Keywords

Long-span language models; Recurrent neural networks; Speech recognition; Decoding

Abstract

This paper deals with approximate inference: a sampling based modeling technique to capture complex dependencies in a language model

Annotation

In this paper, we present strategies to incorporate long context information directly during the first pass decoding and also for the second pass lattice re-scoring in speech recognition systems. Long-span language models that capture complex syntactic and/or semantic information are seldom used in the first pass of large vocabulary continuous speech recognition systems due to the prohibitive increase in the size of the sentence-hypotheses search space. Typically, n-gram language models are used in the first pass to produce N-best lists, which are then re-scored using long-span models. Such a pipeline produces biased first pass output, resulting in sub-optimal performance during re-scoring. In this paper we show that computationally tractable variational approximations of the long-span and complex language models are a better choice than the standard n-gram model for the first pass decoding and also for lattice re-scoring.

Published
2012
Pages
1–16
Journal
Speech Communication, vol. 2012, no. 8, ISSN 0167-6393
Book
Speech Communication
Publisher
Elsevier Science
DOI
UT WoS
000312422900013
EID Scopus
BibTeX
@article{BUT97047,
  author="Anoop {Deoras} and Tomáš {Mikolov} and Stefan {Kombrink} and Kenneth {Church}",
  title="Approximate inference: A sampling based modeling technique to capture complex dependencies in a language model",
  journal="Speech Communication",
  year="2012",
  volume="2012",
  number="8",
  pages="1--16",
  doi="10.1016/j.specom.2012.08.004",
  issn="0167-6393",
  url="http://www.sciencedirect.com/science/article/pii/S0167639312000969#"
}
Back to top