Neural Reasoning with cognitively-inspired Representations
Description: Recently, Bortoletto et al. [1] have proposed IRENE, an Intuitive Reasoning Network that achieves state-of-the-art results on the challenging Baby Intuitions Benchmark (BIB) [2]. The BIB is a common-sense reasoning benchmark that evaluates a models ability to reason about agents' goals, preferences, and actions by observing their behaviour in a grid-world environment. To solve this, IRENE combines a graph neural network for learning agent and world state representations with a transformer to encode the task context.
In this project, you will replace the graph neural network in IRENE with a cognitively-inspired encoding of the state representation, i.e. with Spatial Semantic Pointers (SSPs). SSPs can build a meaningful state representation by creating a spatial map, where objects are bound to their spatial locations. This representation has been shown to improve agents' learning of desired behaviour in navigation tasks [3] and has further desirable properties, such as efficiency and interpretability.
Goal: Goal: Use SSPs [3] to encode state representations (video frames) and perform intuitive reasoning with IRENE [1] on the BIB [2].
Supervisor: Matteo Bortoletto and Anna Penzkofer
Distribution: Distribution: 10% literature review, 70% implementation, 20% analysis
Requirements: : Requirements: Programming in Python, interest in deep learning and cognitive models.
Literature: [1] M. Bortoletto, L. Shi, and A. Bulling. Neural reasoning about agents’ goals, preferences, and actions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 456–464, 2024 Paper link.
[2] K. Gandhi, G. Stojnic, B. M. Lake, and M. Dillon. Baby intuitions benchmark (BIB): Discerning the goals, preferences, and actions of others. In Thirty-Fifth Conference on Neural Information Processing Systems, 2021. Paper link.
[3] B. Komer and C. Eliasmith. Efficient navigation using a scalable, biologically inspired spatial representation. 2020. Paper link.