Publication Details

MixedEmotions: An Open-Source Toolbox for Multimodal Emotion Analysis

BUITELAAR, P.; WOOD, I.; NEGI, S.; ARCAN, M.; MCCRAE, J.; ABELE, A.; ROBIN, C.; ANDRYUSHECHKIN, V.; ZIAD, H.; SAGHA, H.; SCHMITT, M.; SCHULLER, B.; SÁNCHEZ-RADA, J.; IGLESIAS, C.; NAVARRO, C.; GIEFER, A.; HEISE, N.; MASUCCI, V.; DANZA, F.; CATERINO, C.; SMRŽ, P.; HRADIŠ, M.; POVOLNÝ, F.; KLIMEŠ, M.; MATĚJKA, P.; TUMMARELLO, G. MixedEmotions: An Open-Source Toolbox for Multimodal Emotion Analysis. IEEE TRANSACTIONS ON MULTIMEDIA, 2018, vol. 20, no. 9, p. 2454-2465. ISSN: 1520-9210.
Czech title
MixedEmotions: Nástroje pro multimodální emoční analýzu s otevřeným kódem
Type
journal article
Language
English
Authors
BUITELAAR, P.
Wood Ian (FIT)
NEGI, S.
ARCAN, M.
MCCRAE, J.
ABELE, A.
Robin Cecile (FIT)
ANDRYUSHECHKIN, V.
ZIAD, H.
SAGHA, H.
SCHMITT, M.
SCHULLER, B.
SÁNCHEZ-RADA, J.
IGLESIAS, C.
NAVARRO, C.
GIEFER, A.
HEISE, N.
MASUCCI, V.
DANZA, F.
CATERINO, C.
Smrž Pavel, doc. RNDr., Ph.D. (DCGM)
Hradiš Michal, Ing., Ph.D. (DCGM)
Povolný Filip, Ing.
Klimeš Marek, Ing., DiS.
Matějka Pavel, Ing., Ph.D. (DCGM)
TUMMARELLO, G.
URL
Keywords

emotion analysis, open source toolbox, affective computing, linked data, audio processing, text processing, video processing

Abstract

Recently, there is an increasing tendency to embed functionalities for recognizing emotions from user-generated media content in automated systems such as call-centre operations, recommendations, and assistive technologies, providing richer and more informative user and content profiles. However, to date, adding these functionalities was a tedious, costly, and time-consuming effort, requiring identification and integration of diverse tools with diverse interfaces as required by the use case at hand. The MixedEmotions Toolbox leverages the need for such functionalities by providing tools for text, audio, video, and linked data processing within an easily integrable plug-and-play platform. These functionalities include: 1) for text processing: emotion and sentiment recognition; 2) for audio processing: emotion, age, and gender recognition; 3) for video processing: face detection and tracking, emotion recognition, facial landmark localization, head pose estimation, face alignment, and body pose estimation; and 4) for linked data: knowledge graph integration. Moreover, the MixedEmotions Toolbox is open-source and free. In this paper, we present this toolbox in the context of the existing landscape, and provide a range of detailed benchmarks on standard test-beds showing its state-of-the-art performance. Furthermore, three real-world use cases show its effectiveness, namely, emotion-driven smart TV, call center monitoring, and brand reputation analysis.

Published
2018
Pages
2454–2465
Journal
IEEE TRANSACTIONS ON MULTIMEDIA, vol. 20, no. 9, ISSN 1520-9210
DOI
10.1109/TMM.2018.2798287
UT WoS
000442358200017
EID Scopus
2-s2.0-85040988311
BibTeX 
@article{BUT155798,
  author="BUITELAAR, P. and WOOD, I. and NEGI, S. and ARCAN, M. and MCCRAE, J. and ABELE, A. and ROBIN, C. and ANDRYUSHECHKIN, V. and ZIAD, H. and SAGHA, H. and SCHMITT, M. and SCHULLER, B. and SÁNCHEZ-RADA, J. and IGLESIAS, C. and NAVARRO, C. and GIEFER, A. and HEISE, N. and MASUCCI, V. and DANZA, F. and CATERINO, C. and SMRŽ, P. and HRADIŠ, M. and POVOLNÝ, F. and KLIMEŠ, M. and MATĚJKA, P. and TUMMARELLO, G.",
  title="MixedEmotions: An Open-Source Toolbox for Multimodal Emotion Analysis",
  journal="IEEE TRANSACTIONS ON MULTIMEDIA",
  year="2018",
  volume="20",
  number="9",
  pages="2454--2465",
  doi="10.1109/TMM.2018.2798287",
  issn="1520-9210",
  url="https://ieeexplore.ieee.org/document/8269329/?arnumber=8269329"
}
Back to top