In this task, we identify a challenge that is reflective of linguistic and cognitive competencies that humans have when speaking and reasoning. Particularly, given the intuition that textual and visual information mutually inform each other for semantic reasoning, we formulate a Competence-based Question Answering challenge, designed to involve rich semantic annotation and aligned text-video objects. The task is to answer questions from a collection of cooking recipes and videos, where each question belongs to a “question family” reflecting a specific reasoning competence. The data and task result is publicly available.
2022, Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), Pages 1244-1255
SemEval-2022 Task 9: R2VQ – Competence-based Multimodal Question Answering (04b Atto di convegno in volume)
Tu Jingxuan, Holderness Eben, Maru Marco, Conia Simone, Rim Kyeongmin, Lynch Kelley, Brutti Richard, Navigli Roberto, Pustejovsky James
Gruppo di ricerca: Natural Language Processing