Multiple Intelligent Conversation Agent Sevices for Reception, Management and Integration of Third Country Nationals

Czech title

Vícenásobné služby inteligentního konverzačního agenta pro přijetí, řízení a integraci občanů třetích zemí v EU

Type

grant

Keywords

migration, social issues, public sector innovation, personalized embodied conversation agents, language teaching, social and societal inclusion, decision support, multilingual language technologies, virtual/augmented reality, knowledge processing.

Abstract

The objective of WELCOME is to research and develop intelligent technologies for support of the reception and integration of migrants in Europe. Unlike the majority of the projects that address the problem of migrant integration by frontends with "one-for-all" interfaces for information acquisition and services, WELCOME will offer a personalized and psychologically and socially competent solution for both migrants and public administrations. It will develop immersive and intelligent services, in which embodied intelligent multilingual agents will act as dedicated personalized assistants of migrants in contexts of registration, orientation, language teaching, civic education, and social and societal inclusion. The agents will be personalized in the sense that they will dynamically adapt their interaction behavior to the topic, given context and the profile of the interlocutor. To provide "real-life" experience and thus deeper and swifter integration, immersive virtual and augmented reality technologies will be used. For public administrations, decision support technologies that will draw upon visual analytics and semantic reasoning techniques will be developed. To achieve its objective, WELCOME will innovate in the areas of human - multiple service-oriented agent interaction, multilingual spoken language technologies, knowledge processing, immersive personalized education and social inclusion technologies, and decision support strategies. WELCOMEs solution will be validated and its portability to different European contexts demonstrated in three pilot use cases. Both WELCOME as a whole and its individual technologies are expected to have a very significant societal and economic impact. WELCOME counts with a highly competent Consortium of 15 partners: 6 research institutions, 3 ICT companies, and 6 entities related to migrant reception and integration. To increase its impact and dissemination, WELCOME involves the International Migrant Organization as subcontractor.

Team members

Černocký Jan, prof. Dr. Ing. (DCGM) – research leader
Burget Lukáš, doc. Ing., Ph.D. (DCGM)
Karafiát Martin, Ing., Ph.D. (DCGM)
Plchot Oldřich, Ing., Ph.D. (DCGM)
Žižka Josef, Ing. (DCGM)

Support

[img]
This project has received funding from the European Unions Horizon 2020 research and innovation programme under grant agreement No 870930.

Publications

2023

YUSUF, B.; ČERNOCKÝ, J.; SARAÇLAR, M. End-to-End Open Vocabulary Keyword Search With Multilingual Neural Representations. IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH AND LANGUAGE PROCESSING, 2023, vol. 31, no. 08, p. 3070-3080. ISSN: 2329-9290. Detail

2022

EGOROVA, E.; VYDANA, H.; BURGET, L.; ČERNOCKÝ, J. Spelling-Aware Word-Based End-to-End ASR. IEEE SIGNAL PROCESSING LETTERS, 2022, vol. 29, no. 29, p. 1729-1733. ISSN: 1558-2361. Detail
ONDEL YANG, L.; YUSUF, B.; BURGET, L.; SARAÇLAR, M. Non-Parametric Bayesian Subspace Models for Acoustic Unit Discovery. IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH AND LANGUAGE PROCESSING, 2022, vol. 30, no. 5, p. 1902-1917. ISSN: 2329-9290. Detail
ŠVEC, J.; ŽMOLÍKOVÁ, K.; KOCOUR, M.; DELCROIX, M.; OCHIAI, T.; MOŠNER, L.; ČERNOCKÝ, J. Analysis of impact of emotions on target speech extraction and speech separation. In Proceedings of The 17th International Workshop on Acoustic Signal Enhancement (IWAENC 2022). Bamberg: IEEE Signal Processing Society, 2022. p. 1-5. ISBN: 978-1-6654-6867-1. Detail

2021

EGOROVA, E.; VYDANA, H.; BURGET, L.; ČERNOCKÝ, J. Out-of-Vocabulary Words Detection with Attention and CTC Alignments in an End-to-End ASR System. In Proceedings Interspeech 2021. Proceedings of Interspeech. Brno: International Speech Communication Association, 2021. p. 2901-2905. ISSN: 1990-9772. Detail
WANNER, L.; KLUSCH, M.; MAVROPOULOS, A.; JAMIN, E.; MARIN PUCHADES, V.; CASAMAYOR, G.; ČERNOCKÝ, J.; EGOROVA, E. Towards a Versatile Intelligent Conversational Agent as Personal Assistant for Migrants. In The PAAMS Collection. PAAMS 2021: Advances in Practical Applications of Agents, Multi-Agent Systems, and Social Good. Lecture Notes in Computer Science. Lecture Notes in Computer Science book series. Salamanca: Springer International Publishing, 2021. p. 316-327. ISBN: 978-3-030-85739-4. ISSN: 0302-9743. Detail
YUSUF, B.; GOK, A.; GUNDOGDU, B.; SARAÇLAR, M. End-to-End Open Vocabulary Keyword Search. In Proceedings Interspeech 2021. Proceedings of Interspeech. Brno: International Speech Communication Association, 2021. p. 4388-4392. ISSN: 1990-9772. Detail
YUSUF, B.; ONDEL YANG, L.; BURGET, L.; ČERNOCKÝ, J.; SARAÇLAR, M. A Hierarchical Subspace Model for Language-Attuned Acoustic Unit Discovery. In ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Toronto, Ontario: IEEE Signal Processing Society, 2021. p. 3710-3714. ISBN: 978-1-7281-7605-5. Detail