Publication Details
Analysis of Multilingual Sequence-to-Sequence Speech Recognition Systems
Baskar Murali Karthick, Ing., Ph.D.
Watanabe Shinji
HORI, T.
Wiesner Matthew, PhD.
Černocký Jan, prof. Dr. Ing. (DCGM)
multilingual ASR, sequence-to-sequence, language-transfer, multilingual
bottle-neck feature
This paper investigates the applications of various multilingual approaches
developed in conventional deep neural network - hidden Markov model (DNN-HMM)
systems to sequence-tosequence (seq2seq) automatic speech recognition (ASR). We
employ a joint connectionist temporal classification-attention network as our
base model. Our main contribution is separated into two parts. First, we
investigate the effectiveness of the seq2seq model with stacked multilingual
bottle-neck features obtained from a conventional DNN-HMM system on the Babel
multilingual speech corpus. Second, we investigate the effectiveness of transfer
learning from a pre-trained multilingual seq2seq model with and without the
target language included in the original multilingual training data. In this
experiment, we also explore various architectures and training strategies of the
multilingual seq2seq model by making use of knowledge obtained in the DNN-HMM
based transfer-learning. Although both approaches significantly improved the
performance from a monolingual seq2seq baseline, interestingly, we found the
multilingual bottle-neck features to be superior to multilingual models with
transfer learning. This finding suggests that we can efficiently combine the
benefits of the DNN-HMM system with the seq2seq system through multilingual
bottle-neck feature techniques.
@inproceedings{BUT159995,
author="KARAFIÁT, M. and BASKAR, M. and WATANABE, S. and HORI, T. and WIESNER, M. and ČERNOCKÝ, J.",
title="Analysis of Multilingual Sequence-to-Sequence Speech Recognition Systems",
booktitle="Proceedings of Interspeech",
year="2019",
journal="Proceedings of Interspeech",
volume="2019",
number="9",
pages="2220--2224",
publisher="International Speech Communication Association",
address="Graz",
doi="10.21437/Interspeech.2019-2355",
issn="1990-9772",
url="https://www.isca-speech.org/archive/Interspeech_2019/pdfs/2355.pdf"
}