Text Augmentation for Language Models in High Error Recognition Scenario

In this paper, we explore several data augmentation strategiesfor training of language models for speech recognition. Wecompare augmentation based on global error statistics withone based on unigram statistics of ASR errors and with labelsmoothingand its sampled variant. Additionally, we investigatethe stability and the predictive power of perplexity estimatedon augmented data. Despite being trivial, augmentation drivenby global substitution, deletion and insertion rates achieves thebest rescoring results. On the other hand, even though the associatedperplexity measure is stable, it gives no better predictionof the final error rate than the vanilla one. Our best augmentationscheme increases the WER improvement from second-passrescoring from 1.1% to 1.9% absolute on the CHiMe-6 challenge.