Publication Details
Rethinking the Objectives of Extractive Question Answering
QA, extractive QA, independent objective, joint objective, compound objective
This work demonstrates that using the objective with independence assumption for
modelling the span probability P (a_s , a_e ) = P (a_s )P (a_e) of span starting
at position a_s and ending at position a_e has adverse effects. Therefore we
propose multiple approaches to modelling joint probability P (a_s , a_e)
directly. Among those, we propose a compound objective, composed from the joint
probability while still keeping the objective with independence assumption as an
auxiliary objective. We find that the compound objective is consistently superior
or equal to other assumptions in exact match. Additionally, we identified common
errors caused by the assumption of independence and manually checked the
counterpart predictions, demonstrating the impact of the compound objective on
the real examples. Our findings are supported via experiments with three
extractive QA models (BIDAF, BERT, ALBERT) over six datasets and our code,
individual results and manual analysis are available online.
@inproceedings{BUT175858,
author="Martin {Fajčík} and Josef {Jon} and Pavel {Smrž}",
title="Rethinking the Objectives of Extractive Question Answering",
booktitle="Proceedings of the 3rd Workshop on Machine Reading for Question Answering",
year="2021",
series="Proceedings of the 3rd Workshop on Machine Reading for Question Answering",
pages="14--27",
publisher="Association for Computational Linguistics",
address="Punta Cana",
isbn="978-1-954085-95-4",
url="https://aclanthology.org/2021.mrqa-1.2/"
}