Service restrictions from February 12-22, 2026—more information on the University Library website

Result: Exploration of Rationale-Extraction Methods for Closed-Domain Question Answering with a New Sentence-Level Rationale Dataset

Title:
Exploration of Rationale-Extraction Methods for Closed-Domain Question Answering with a New Sentence-Level Rationale Dataset
Authors:
Pirenne, Lize, Mokeddem, Samy, Ernst, Damien, Louppe, Gilles
Contributors:
Montefiore Institute - Montefiore Institute of Electrical Engineering and Computer Science - ULiège
Source:
urn:isbn:978-3-031-97144-0
urn:isbn:978-3-031-97143-3
Natural Language Processing and Information Systems, 3-13 (2025-07-01); 30th Annual International Conference on Natural Language & Information Systems (NLDB 2025), kanazawa, Japan [JP], 4 July 2025 - 6 July 2025
Publisher Information:
Springer Nature, 2025.
Publication Year:
2025
Subject Terms:
Large Language Models, Rationale Extraction, Classifier, Natural Language Processing, Reinforcement Learning, Fine Tuning, Embedding, Engineering, computing & technology, Computer science, Ingénierie, informatique & technologie, Sciences informatiques
Document Type:
Conference conference paper<br />http://purl.org/coar/resource_type/c_5794<br />conferenceObject<br />peer reviewed
Language:
English
Relation:
https://www.jaist.ac.jp/event/nldb2025/guide.html; https://www.jaist.ac.jp/event/nldb2025/guide.html; https://huggingface.co/datasets/Inversta/rationale-databricks-dolly-cqa
DOI:
10.1007/978-3-031-97144-0_1
Access URL:
https://orbi.uliege.be/handle/2268/322654
Rights:
open access
http://purl.org/coar/access_right/c_abf2
info:eu-repo/semantics/openAccess
Accession Number:
edsorb.322654
Database:
ORBi

Further Information

In this paper, we address the problem of Rationale Extraction (RE) from Natural Language Processing: given a context (C), a related question (Q) and its answer (A), the task is to find the best sentence-level rationale (R*). This rationale is loosely defined as being the subset of sentences of the context C such that producing A would require at least R*. We have constructed a dataset where each entry is composed of the four terms (C, Q, A, R*) to explore different methods in the particular case where the answer is one or multiple full sentences. The methods studied are based on TF-IDF scores, embedding similarity, classifiers and attention and have been evaluated using a sentence overlap metric akin to the Intersection over Union (IoU). Results show that the best scores were achieved by the classifier-based approach with the nuance of a better scaling with the attention-based method as the size of the context increases, which is a challenge for all other methods. We also show that generating A significantly decreases the performance of the attention-based method, but training the model to generate A can improve the results, linking the ability to generate with the accomplishment of the task.
ARIAC by DW4AI