Publication Details
Leveraging Self-Supervised Learning for Speaker Diarization
Landini Federico Nicolás (DCGM FIT BUT)
Rohdin Johan A., Dr. (DCGM FIT BUT)
Silnova Anna, MSc., Ph.D. (DCGM FIT BUT)
Diez Sánchez Mireia, M.Sc., Ph.D. (DCGM FIT BUT)
Burget Lukáš, doc. Ing., Ph.D. (DCGM FIT BUT)
Speaker diarization, data scarcity, WavLM, Pyannote, far-field meeting data
End-to-end neural diarization has evolved considerably over
the past few years, but data scarcity is still a major obstacle for further
improvements. Self-supervised learning methods such as WavLM have
shown promising performance on several downstream tasks, but their
application on speaker diarization is somehow limited. In this work,
we explore using WavLM to alleviate the problem of data scarcity
for neural diarization training. We use the same pipeline as Pyannote
and improve the local end-to-end neural diarization with WavLM and
Conformer. Experiments on far-field AMI, AISHELL-4, and AliMeeting
datasets show that our method substantially outperforms the Pyannote
baseline and achieves new state-of-the-art results on AMI and AISHELL-
4, respectively. In addition, by analyzing the system performance under
different data quantity scenarios, we show that WavLM representations
are much more robust against data scarcity than filterbank features,
enabling less data hungry training strategies. Furthermore, we found
that simulated data, usually used to train end-to-end diarization models,
does not help when using WavLM in our experiments. Additionally, we
also evaluate our model on the recent CHiME8 NOTSOFAR-1 task where
it achieves better performance than the Pyannote baseline. Our source
code is publicly available at https://github.com/BUTSpeechFIT/DiariZen.
@INPROCEEDINGS{FITPUB13519,
author = "Jiangyu Han and Nicol\'{a}s Federico Landini and A. Johan Rohdin and Anna Silnova and Mireia S\'{a}nchez Diez and Luk\'{a}\v{s} Burget",
title = "Leveraging Self-Supervised Learning for Speaker Diarization",
pages = "1--5",
booktitle = "Proceedings of ICASSP 2025",
year = 2025,
location = "Hyderabad, IN",
publisher = "IEEE Biometric Council",
ISBN = "979-8-3503-6874-1",
doi = "10.1109/ICASSP49660.2025.10889475",
language = "english",
url = "https://www.fit.vut.cz/research/publication/13519"
}