Publication Details
Claim-Dissector: An Interpretable Fact-Checking System with Joint Re-ranking and Veracity Prediction
Motlíček Petr, doc. Ing., Ph.D. (DCGM)
Smrž Pavel, doc. RNDr., Ph.D. (DCGM)
fact-checking, open-domain fact-checking, claim, claim-dissector, dissector, fine-grained retrieval, coarse-grained supervision, interpretability, interpretable retrieval, evidence-grounded prediction, verification, fact verification, veracity assessement
We present Claim-Dissector: a novel latent variable model for fact-checking and analysis, which given a claim and a set of retrieved evidence jointly learns to identify: (i) the relevant evidences to the given claim (ii) the veracity of the claim. We propose to disentangle the per-evidence relevance probability and its contribution to the final veracity probability in an interpretable way - the final veracity probability is proportional to a linear ensemble of per-evidence relevance probabilities. In this way, the individual contributions of evidences towards the final predicted probability can be identified. In per-evidence relevance probability, our model can further distinguish whether each relevant evidence is supporting (S) or refuting (R) the claim. This allows to quantify how much the S/R probability contributes to final verdict or to detect disagreeing evidence. Despite its interpretable nature, our system achieves results competetive with state-of-the-art on the FEVER dataset, as compared to typical two-stage system pipelines, while using significantly fewer parameters. Furthermore, our analysis shows that our model can learn fine-grained relevance cues while using coarse-grained supervision and we demonstrate it in 2 ways. (i) We show that our model can achieve competitive sentence recall while using only paragraph-level relevance supervision. (ii) Traversing towards the finest granularity of relevance, we show that our model is capable of identifying relevance at the token level. To do this, we present a new benchmark TLR-FEVER focusing on token-level interpretability - humans annotate tokens in relevant evidences they considered essential when making their judgment. Then we measure how similar are these annotations to the tokens our model is focusing on.
@inproceedings{BUT185594,
author="Martin {Fajčík} and Petr {Motlíček} and Pavel {Smrž}",
title="Claim-Dissector: An Interpretable Fact-Checking System with Joint Re-ranking and Veracity Prediction",
booktitle="Findings of the Association for Computational Linguistics: ACL 2023",
year="2023",
series="ACL",
volume="2023",
pages="10184--10205",
publisher="Association for Computational Linguistics",
address="Toronto",
doi="10.18653/v1/2023.findings-acl.647",
isbn="978-1-959429-62-3",
url="https://aclanthology.org/2023.findings-acl.647/"
}