Project Details
Automatic collection and processing of voice data from air-traffic communications
Project Period: 1. 11. 2019 – 28. 2. 2022
Project Type: grant
Agency: Evropská unie
Program: Horizon 2020
air-traffic management, automatic speech recognition, signal processing, legal
and ethical framework
Developing machine learning solutions for air-traffic control applications is
a challenging task. Besides an expert knowledge, large amount of data for robust
performance as well as for validation and verification is typically required. If
funded, ATCO2 will deliver a unique platform enabling to collect, store,
pre-process and share voice communications data recorded from real world
air-traffic control data. The project aims at accessing data from two sources:
(a) from certified ADS-B datalinks aligned with a surveillance technology, and
(b) directly from air-traffic controllers offered to the project by several air
navigation service providers. The technical development will be centred around
the ATCO2 platform, built on an existing and extensively used solution of
opensky-network partner, ensuring sustainability of the platform after the end of
the project. Current platform collects periodically broadcasted aircraft
information through a network of ADS-B receivers operated around the globe,
further stored at a server. In ATCO2, existing platform will be extended to allow
collection, storage and pre-processing of voice communications, and time/position
aligned with other aircraft information. Unlike previous works, we will target
both channels, i.e. spoken commands issued by air-traffic controllers, and
confirmation provided by pilots. In addition to broadcasted data, ATCO2 will have
an access to voice recordings from air navigation service providers, namely
Austrocontrol. This data will simulate other source of speech recordings
(specifically archives), complementing real-time voice communication. The ATCO2
platform will be enhanced by the latest speech pre-processing and machine
learning technologies, mostly based on deep learning. Besides automatic
segmentation (e.g. er speaker, accent, specific command), robust automatic speech
recognition system will be implemented and integrated through RESTful API
allowing to automatically transcribe voice communications.
Kocour Martin, Ing. (DCGM)
Pulugundla Bhargav, M.Sc.
Veselý Karel, Ing., Ph.D. (DCGM)
Žižka Josef, Ing. (DCGM)
2023
- ZULUAGA-GOMEZ, J.; NIGMATULINA, I.; PRASAD, A.; MOTLÍČEK, P.; KHALIL, D.; MADIKERI, S.; TART, A.; SZŐKE, I.; LENDERS, V.; RIGAULT, M.; CHOUKRI, K. Lessons Learned in Transcribing 5000 h of Air Traffic Control Communications for Robust Automatic Speech Understanding. Aerospace, 2023, vol. 2023, no. 10,
p. 1-33. ISSN: 2226-4310. Detail - ZULUAGA-GOMEZ, J.; PRASAD, A.; NIGMATULINA, I.; SARFJOO, S.; MOTLÍČEK, P.; KLEINERT, M.; HELMKE, H.; OHNEISER, O.; ZHAN, Q. How Does Pre-Trained Wav2Vec 2.0 Perform on Domain-Shifted ASR? an Extensive Benchmark on Air Traffic Control Communications. In IEEE Spoken Language Technology Workshop, SLT 2022 - Proceedings. Doha: IEEE Signal Processing Society, 2023.
p. 205-212. ISBN: 978-1-6654-7189-3. Detail
2022
- BLATT, A.; KOCOUR, M.; VESELÝ, K.; SZŐKE, I.; KLAKOW, D. Call-Sign Recognition and Understanding for Noisy Air-Traffic Transcripts Using Surveillance Information. In ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings. Singapore: IEEE Signal Processing Society, 2022.
p. 8357-8361. ISBN: 978-1-6654-0540-9. Detail - KOCOUR, M.; ŽMOLÍKOVÁ, K.; ONDEL YANG, L.; ŠVEC, J.; DELCROIX, M.; OCHIAI, T.; BURGET, L.; ČERNOCKÝ, J. Revisiting joint decoding based multi-talker speech recognition with DNN acoustic model. In Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. Proceedings of Interspeech. Incheon: International Speech Communication Association, 2022.
p. 4955-4959. ISSN: 1990-9772. Detail - NIGMATULINA, I.; ZULUAGA-GOMEZ, J.; PRASAD, A.; SARFJOO, S.; MOTLÍČEK, P. A Two-Step Approach to Leverage Contextual Data: Speech Recognition in Air-Traffic Communications. In ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings. Singapore: IEEE Signal Processing Society, 2022.
p. 6282-6286. ISBN: 978-1-6654-0540-9. Detail - PRASAD, A.; ZULUAGA-GOMEZ, J.; MOTLÍČEK, P.; SARFJOO, S.; NIGMATULINA, I.; OHNEISER, O.; HELMKE, H. Grammar Based Speaker Role Identification for Air Traffic Control Speech Recognition. Proceedings of the 12th SESAR Innovation Days. Budapest: 2022.
p. 1-9. Detail - PRASAD, A.; ZULUAGA-GOMEZ, J.; MOTLÍČEK, P.; SARFJOO, S.; NIGMATULINA, I.; VESELÝ, K. Speech and Natural Language Processing Technologies for Pseudo-Pilot Simulator. Proceedings of the 12th SESAR Innovation Days. Budapest: 2022.
p. 1-9. Detail
2021
- KOCOUR, M.; CÁMBARA, G.; LUQUE, J.; BONET, D.; FARRÚS, M.; KARAFIÁT, M.; VESELÝ, K.; ČERNOCKÝ, J. BCN2BRNO: ASR System Fusion for Albayzin 2020 Speech to Text Challenge. Proceedings of IberSPEECH 2021. Vallaloid: International Speech Communication Association, 2021.
p. 113-117. Detail - KOCOUR, M.; VESELÝ, K.; BLATT, A.; ZULUAGA-GOMEZ, J.; SZŐKE, I.; ČERNOCKÝ, J.; KLAKOW, D.; MOTLÍČEK, P. Boosting of Contextual Information in ASR for Air-Traffic Call-Sign Recognition. In Proceedings Interspeech 2021. Proceedings of Interspeech. Brno: International Speech Communication Association, 2021.
p. 3301-3305. ISSN: 1990-9772. Detail - KOCOUR, M.; VESELÝ, K.; SZŐKE, I.; KESIRAJU, S.; ZULUAGA-GOMEZ, J.; BLATT, A.; PRASAD, A.; NIGMATULINA, I.; MOTLÍČEK, P.; KLAKOW, D.; TART, A.; KOLČÁREK, P.; ČERNOCKÝ, J.; CEVENINI, C.; CHOUKRI, K.; RIGAULT, M.; LANDIS, F.; SARFJOO, S. Automatic Processing Pipeline for Collecting and Annotating Air-Traffic Voice Communication Data. In Proceedings of 9th OpenSky Symposium 2021, OpenSky Network, Brussels, Belgium. Proceedings. Brussels: MDPI, 2021.
p. 1-10. ISSN: 2504-3900. Detail - SZŐKE, I.; KESIRAJU, S.; NOVOTNÝ, O.; KOCOUR, M.; VESELÝ, K.; ČERNOCKÝ, J. Detecting English Speech in the Air Traffic Control Voice Communication. In Proceedings Interspeech 2021. Proceedings of Interspeech. Brno: International Speech Communication Association, 2021.
p. 3286-3290. ISSN: 1990-9772. Detail - VYDANA, H.; KARAFIÁT, M.; ŽMOLÍKOVÁ, K.; BURGET, L.; ČERNOCKÝ, J. Jointly Trained Transformers Models for Spoken Language Translation. In ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Toronto, Ontario: IEEE Signal Processing Society, 2021.
p. 7513-7517. ISBN: 978-1-7281-7605-5. Detail - ZULUAGA-GOMEZ, J.; NIGMATULINA, I.; PRASAD, A.; MOTLÍČEK, P.; VESELÝ, K.; KOCOUR, M.; SZŐKE, I. Contextual Semi-Supervised Learning: An Approach to Leverage Air-Surveillance and Untranscribed ATC Data in ASR Systems. In Proceedings Interspeech 2021. Proceedings of Interspeech. Brno: International Speech Communication Association, 2021.
p. 3296-3300. ISSN: 1990-9772. Detail
2020
- ZULUAGA-GOMEZ, J.; MOTLÍČEK, P.; ZHAN, Q.; VESELÝ, K.; BRAUN, R. Automatic Speech Recognition Benchmark for Air-Traffic Communications. In Proceedings of Interspeech 2020. Proceedings of Interspeech. Shanghai: International Speech Communication Association, 2020.
p. 2297-2301. ISSN: 1990-9772. Detail - ZULUAGA-GOMEZ, J.; VESELÝ, K.; BLATT, A.; MOTLÍČEK, P.; KLAKOW, D.; TART, A.; SZŐKE, I.; PRASAD, A.; SARFJOO, S.; KOLČÁREK, P.; KOCOUR, M.; ČERNOCKÝ, J.; CEVENINI, C.; CHOUKRI, K.; RIGAULT, M.; LANDIS, F. Automatic Call Sign Detection: Matching Air Surveillance Data with Air Traffic Spoken Communications. Proceedings of the 8th OpenSky Symposium 2020. Proceedings. Brusel: MDPI, 2020.
p. 1-10. ISSN: 2504-3900. Detail