Perceptual User Interfaces Logo
University of Stuttgart Logo

MQA-RC-Reading Comprehension

Abstract

We present a novel reading comprehension eye tracking dataset - MQA-RC - which allows researchers to observe changes in reading behavior in three comprehension tasks. Our extension uses 32 movie plots, with corresponding question-answer (QA) pairs, from the benchmark MovieQA dataset (Tapaswi et al., 2015). To the best of our knowledge, this is the first eye tracking dataset over a QA corpus and thus provides a gold standard to compare and synchronize model versus human visual attention in machine comprehension tasks.

The full dataset can be requested by filling out an EULA license agreement and send the agreement to pui-office@vis.uni-stuttgart.de.

Contact: Mrs. Daniela Milanese

The data is only to be used for non-commercial scientific purposes. If you use this dataset in a scientific publication, please cite the following paper:

  1. Interpreting Attention Models with Human Visual Attention in Machine Reading Comprehension

    Interpreting Attention Models with Human Visual Attention in Machine Reading Comprehension

    Ekta Sood, Simon Tannert, Diego Frassinelli, Andreas Bulling, Ngoc Thang Vu

    Proc. ACL SIGNLL Conference on Computational Natural Language Learning (CoNLL), pp. 12-25, 2020.

    Abstract Links BibTeX Project