Publication Details

Semi-supervised Sequence-to-sequence ASR using Unpaired Speech and Text

BASKAR, M.; WATANABE, S.; ASTUDILLO, R.; HORI, T.; BURGET, L.; ČERNOCKÝ, J. Semi-supervised Sequence-to-sequence ASR using Unpaired Speech and Text. In Proceedings of Interspeech. Proceedings of Interspeech. Graz: International Speech Communication Association, 2019. p. 3790-3794. ISSN: 1990-9772.
Czech title
ASR založené na převodu sekvencí na sekvence s lehkou supervizí využívající nesouvisející řečová a textová data
Type
conference paper
Language
English
Authors
Baskar Murali Karthick, Ing., Ph.D.
Watanabe Shinji (FIT)
ASTUDILLO, R.
HORI, T.
Burget Lukáš, doc. Ing., Ph.D. (DCGM)
Černocký Jan, prof. Dr. Ing. (DCGM)
URL
Keywords

Sequence-to-sequence, end-to-end, ASR, TTS, semi-supervised, unsupervised, cycle consistency

Abstract

Sequence-to-sequence automatic speech recognition (ASR) models require large quantities of data to attain high performance. For this reason, there has been a recent surge in interest for unsupervised and semi-supervised training in such models. This work builds upon recent results showing notable improvements in semi-supervised training using cycle-consistency and related techniques. Such techniques derive training procedures and losses able to leverage unpaired speech and/or text data by combining ASR with Text-to-Speech (TTS) models. In particular, this work proposes a new semi-supervised loss combining an end-to-end differentiable ASR!TTS loss with TTS!ASR loss. The method is able to leverage both unpaired speech and text data to outperform recently proposed related techniques in terms of %WER. We provide extensive results analyzing the impact of data quantity and speech and text modalities and show consistent gains across WSJ and Librispeech corpora. Our code is provided in ESPnet to reproduce the experiments.

Published
2019
Pages
3790–3794
Journal
Proceedings of Interspeech, vol. 2019, no. 9, ISSN 1990-9772
Proceedings
Proceedings of Interspeech
Publisher
International Speech Communication Association
Place
Graz
DOI
10.21437/Interspeech.2019-3167
UT WoS
000831796403198
EID Scopus
2-s2.0-85074698215
BibTeX 
@inproceedings{BUT159996,
  author="BASKAR, M. and WATANABE, S. and ASTUDILLO, R. and HORI, T. and BURGET, L. and ČERNOCKÝ, J.",
  title="Semi-supervised Sequence-to-sequence ASR using Unpaired Speech and Text",
  booktitle="Proceedings of Interspeech",
  year="2019",
  journal="Proceedings of Interspeech",
  volume="2019",
  number="9",
  pages="3790--3794",
  publisher="International Speech Communication Association",
  address="Graz",
  doi="10.21437/Interspeech.2019-3167",
  issn="1990-9772",
  url="https://www.isca-speech.org/archive/Interspeech_2019/pdfs/3167.pdf"
}
Back to top