Contextual Semi-Supervised Learning: An Approach to Leverage Air-Surveillance and Untranscribed ATC Data in ASR Systems

Air traffic management and specifically air-traffic control (ATC)rely mostly on voice communications between Air Traffic Controllers(ATCos) and pilots. In most cases, these voice communicationsfollow a well-defined grammar that could be leveragedin Automatic Speech Recognition (ASR) technologies. Thecallsign used to address an airplane is an essential part of allATCo-pilot communications. We propose a two-step approachto add contextual knowledge during semi-supervised training toreduce the ASR system error rates at recognizing the part of theutterance that contains the callsign. Initially, we represent in aWFST the contextual knowledge (i.e. air-surveillance data) ofan ATCo-pilot communication. Then, during Semi-SupervisedLearning (SSL) the contextual knowledge is added by secondpassdecoding (i.e. lattice re-scoring). Results show that unseendomains (e.g. data from airports not present in the supervisedtraining data) are further aided by contextual SSL whencompared to standalone SSL. For this task, we introduce theCallsign Word Error Rate (CA-WER) as an evaluation metric,which only assesses ASR performance of the spoken callsignin an utterance. We obtained a 32.1% CA-WER relative improvementapplying SSL with an additional 17.5% CA-WERimprovement by adding contextual knowledge during SSL on achallenging ATC-based test set gathered from LiveATC.