End-to-End Open Vocabulary Keyword Search

Recently, neural approaches to spoken content retrieval have becomepopular. However, they tend to be restricted in their vocabularyor in their ability to deal with imbalanced test settings.These restrictions limit their applicability in keyword search,where the set of queries is not known beforehand, and wherethe system should return not just whether an utterance containsa query but the exact location of any such occurrences.In this work, we propose a model directly optimized for keywordsearch. The model takes a query and an utterance as inputand returns a sequence of probabilities for each frame of theutterance of the query having occurred in that frame. Experimentsshow that the proposed model not only outperforms similarend-to-end models on a task where the ratio of positive andnegative trials is artificially balanced, but it is also able to dealwith the far more challenging task of keyword search with itsinherent imbalance. Furthermore, using our system to rescorethe outputs an LVCSR-based keyword search system leads tosignificant improvements on the latter.