Multimodal Human-Machine Interfaces for Morphing Computerized Mats with Embodied Sensing and Artificial Intelligence

grant

Human-machine interaction, multimodal processing, novel manipulation, usability test

The MOZART project aims to advance the state of the art in robotic handling and manipulation of soft and heterogeneous objects. Its approach is a radical departure from existing technologies as it develops a new concept based on deformable manipulation surfaces supported by AI-powered control and learning tools. While the running project primarily addresses the development of hardware and control algorithms, human-machine interaction is less explored. The role of the new partner - Brno University of Technology, Czechia (BUT), in the proposed project extension is mainly to research the field of Human-Machine Interaction (HMI), explicitly identifying user roles, needs, and necessary interface types, iteratively developing prototypes, and performing evaluation. Usability testing will be carried out in multiple stages of development, including aspects of the social science perspective (with the help of SSH research experts from partner TCD), to assess the system's natural fit with diverse operators. Various input and output modalities and their combinations will be evaluated, including the manipulation surface itself and augmented and virtual reality. Machine learning may be used to achieve the adaptivity of the interface. Various types of interfaces that will be developed will be backed up by a cloudenabled server, acting as a data platform, enabling data sharing between users and offloading computationally intensive tasks. BUT aims to contribute to MOZART's goal of automating the food industry by designing effective user interfaces that enhance system acceptance, leveraging the team's extensive experience in this field.

Smrž Pavel, doc. RNDr., Ph.D. (DCGM) – research leader
Bambušek Daniel, Ing. (DCGM)
Materna Zdeněk, Ing., Ph.D. (DCGM)
Zemčík Pavel, prof. Dr. Ing., dr. h. c. (DCGM)