Publication Details
Probing Self-Supervised Learning Models With Target Speech Extraction
Delcroix Marc
OCHIAI, T.
ASHIHARA, T.
Plchot Oldřich, Ing., Ph.D. (DCGM)
ARAKI, S.
Černocký Jan, prof. Dr. Ing. (DCGM)
Target speech extraction, self-supervised learning, SUPERB
Large-scale pre-trained self-supervised learning (SSL) models have shown
remarkable advancements in speech-related tasks. However, the utilization of
these models in complex multi-talker scenarios, such as extracting a target
speaker in a mixture, is yet to be fully evaluated. In this paper, we introduce
target speech extraction (TSE) as a novel downstream task to evaluate the feature
extraction capabilities of pre-trained SSL models. TSE uniquely requires both
speaker identification and speech separation, distinguishing it from other tasks
in the Speech processing Universal PERformance Benchmark (SUPERB) evaluation.
Specifically, we propose a TSE downstream model composed of two lightweight
task-oriented modules based on the same frozen SSL model. One module functions as
a speaker encoder to obtain target speaker information from an enrollment speech,
while the other estimates the target speaker's mask to extract its speech from
the mixture. Experimental results on the Libri2mix datasets reveal the relevance
of the TSE downstream task to probe SSL models, as its performance cannot be
simply deduced from other related tasks such as speaker verification and
separation.
@inproceedings{BUT189780,
author="PENG, J. and DELCROIX, M. and OCHIAI, T. and ASHIHARA, T. and PLCHOT, O. and ARAKI, S. and ČERNOCKÝ, J.",
title="Probing Self-Supervised Learning Models With Target Speech Extraction",
booktitle="ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings",
year="2024",
pages="535--539",
publisher="IEEE Signal Processing Society",
address="Seoul",
doi="10.1109/ICASSPW62465.2024.10627502",
isbn="979-8-3503-7451-3",
url="https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10627502"
}