Cognitive Science
Design by Amey Zhang
How should an intelligent agent behave in order to best realize their goals? What inferences or actions should they make in order to solve an important computational task? Cognitive science aims to answer these questions at an abstract computational level, using tools from probability theory, statistical inference, and elsewhere.
In this session we will discuss how such optimal behavior should change under different conditions of uncertainty, background knowledge, multiple agents, or constraints on resource. This can be used to understand human behavior in the real world or the lab, as well as build artificial agents that learn robust and generalizable world models from small amounts of data.
Session Chairs
Dr Ruairidh Battleday (Harvard / MIT)
Dr Colin Conwell (Harvard / MIT)
Keynote Talks
Dr Kim Stachenfeld (Google DeepMind)
Dr Ida Momennejad (Microsoft Research)
Invited Talks
Professor Bonan Zhao (Edinburgh)
Professor Lisa-Marie Vortmann (University of Groningen)
Dr Mathias Sablé-Meyer (Harvard / MIT)
Dr. Dan Nicolau (King’s College)
Professor Eran Eldar (Hebrew University of Jerusalem)
Spotlight Talks
Aslan Satary Dizaji (University of Michigan)
Ishan Kalburge (University of Cambridge)
Kai Sandbrink (University of Oxford)
Dr Paxon Frady (UC Berkeley)
Li Wenjie (Carnegie Mellon Univeristy)
Dr Anita Keshmirian (Forward College Berlin)
Keynotes
Microsoft Research
Memory, Planning, and Reasoning in Natural and Artificial Intelligence
I will first show modeling, behavioral, and neuroimaging evidence that a combination of multi-scale predictive representations and replay, mediated by the hippocampus and the prefrontal cortex, capture human memory and planning in health and psychiatric conditions. Inspired by these findings, I will then show how deep learning agents fail at human-like navigation (Xbox games) and how large language models (LLMs) fail at planning and multi-step reasoning (CogEval). I will then show how building multi-agent architectures with LLMs, inspired by the fields of neuroscience or collective cognition, overcomes these failures (MAP) and discuss the need for algorithmic understanding of generative AI (AlgEval). I will conclude with notes on the intelligence of life vs. artificial intelligence, and the road ahead.
Google DeepMind
Discovering symboling cognitive models with LLMs
Symbolic models play a key role in neuroscience and psychology, expressing computationally precise hypotheses about how the brain implements a cognitive process. Identifying an appropriate model typically requires a great deal of effort and ingenuity on the part of a human scientist. Here, we adapt FunSearch [Romera-Paredes et al. (2024)], a recently developed tool that uses Large Language Models (LLMs) in an evolutionary algorithm, to automatically discover symbolic cognitive models that accurately capture human and animal learning. We consider datasets from three species performing a classic reward-learning task that has been the focus of substantial modeling effort, and find that the discovered programs outperform state-of-the-art cognitive models for each. The discovered programs can readily be interpreted as hypotheses about human and animal cognition, instantiating interpretable symbolic learning and decision-making algorithms. Broadly, these results have intriguing takeaways for opportunities and challenges in using LLMs for scientific discovery and theory construction.
Invited Speakers
University of Edinburgh
How Limited Minds Scaffold Complex Concepts
The complexity of modern science and technology has reached far beyond what a mind can compute on its own. This talk explores how we approximate such complex generative models with limited computational resources, focusing particularly on inductive biases and approximate search algorithms. Inductive biases provide structured mental representations that enable rapid, data-efficient inference and generalization, while approximate search algorithms allow us to cache and reuse previous findings, scaffolding increasingly complex mental constructs in principled ways. At the core, we create new things from combining existing things, and use those new constructs as building blocks for future innovations. By presenting a computational model and experimental evidence, I will demonstrate how a resource-rational cache-and-reuse algorithm, when intertwined with computational constraints, gives rise to the diverse and sophisticated representations that intuitively come to people’s mind.
Harvard / MIT
A Language of Thought Theory of Geometric Shape Perception in the Human Brain
The Language of Thought hypothesis posits that mental representations are best understood as program-like objects. In my work, I address questions that arise from taking this hypothesis at face value and unfolding its predictions, ranging from computational accounts to neural correlates. Here, I focus on the perception of geometric shapes. First, using MEG and fMRI, I show that the mere perception of geometric shapes recruits a symbolic mode of visual processing in the brain, beyond the usual visual pathways. Next, I propose a generative language of shapes to account for the complexity of geometric shapes in human cognition and test it. Finally, I outline an algorithm for perception-as-program inference and sketch what a its mechanistic implementation might look like in the human brain.
Hebrew University of Jerusalem
Mood as a Vehicle of Reinforcement Learning
The science of learning and decision-making has largely evolved in isolation from the study of emotions and moods. Yet both fields, I will argue, investigate the same core mechanisms: the fundamental computations of reinforcement learning. By explicitly linking emotions and moods to these computations, we not only gain new leverage to explain real-world behavior but also refine our models to better reflect how humans actually learn and plan. I will illustrate these advances through a large-scale, intensive longitudinal study using a bespoke mobile platform with wearable EEG. The results reveal mood as a vehicle of reinforcement learning and enable mood prediction up to five days in advance. This framework opens new paths for understanding how reinforcement learning functions adaptively, or goes awry.
Spotlight Talks
University of Michigan
Incentives to Build Houses, Trade Houses, or Trade House Building Skills in Simulated Worlds under Various Governing Systems or Institutions: Comparing Multi-agent Reinforcement Learning to Generative Agent-based Model
Social institutions, such as governing systems, shape human motivation and behavior. Traditionally, economists have studied these effects using mathematical models and field experiments. However, advances in artificial intelligence (AI) now allow for in-silico simulations. This study explores governance structures using two AI frameworks: the AI-Economist (a multi-agent reinforcement learning framework) and the Concordia, a generative agent-based model (GABM). The AI-Economist framework is extended to allow agents to trade houses and house-building skills, along with implementing a voting mechanism. This results in three governance types—Full-Libertarian, Semi-Libertarian/Utilitarian, and Full-Utilitarian. The Semi-Libertarian/Utilitarian system is further categorized into Inclusive, Arbitrary, and Extractive institutions. Findings suggest that under the Semi-Libertarian/Utilitarian system, house-building and skill-trading are more prevalent, particularly in the Inclusive institution. Similarly, the Concordia GABM framework, modified to include inventory, skills, voting, and taxation, generates the same three governance types. Results indicate that when prioritizing equality, Full-Utilitarian governance encourages house-building and skill-trading. However, when productivity is emphasized, Full-Libertarian governance leads to higher activity in building, trading houses, and skill exchange. Overall, this exploratory study compares the effectiveness of MARL and GABM in modeling social behaviors under different governing systems. While the findings provide valuable insights, further validation is necessary.
University of Cambridge
Invariant probabilistic representations in neural networks
While mounting evidence indicates that human decision-making follows Bayesian principles, the underlying neural computations remain unknown. Recent work suggests that probabilistic representations arise naturally in neural networks trained with non-probabilistic objectives. However, previous decoding approaches only assessed whether posteriors were decodable from neural activity, i.e., the specificity of the neural code, without testing whether these representations discarded irrelevant input information, i.e., invariance. Thus, they could not distinguish truly probabilistic representations from trivial re-representations of inputs. Here, we formalize this distinction through a novel information-bottleneck requirement: a network’s internal representation should maximize information about relevant posteriors while minimizing information about inputs. The underlying insight is that task-relevant posteriors should be decodable from the hidden layer activities of a network that uses posterior uncertainty to behave optimally, but if inputs are also decodable, the network is not meaningfully transforming those inputs into a usable code for downstream computations.
Using this approach, we demonstrate that when feed-forward networks are trained on classical cue combination tasks without probabilistic objectives, their hidden layer activities robustly encode task-relevant posteriors but fail to adequately compress input representations. Interestingly, we find that these networks exhibit two-stage learning profiles: an early stage that preserves compression, followed by a later stage that discards compression in favor of stronger posterior decodability. Our work provides a general framework to assess both specificity and invariance in probabilistic neural codes and, thus, lays the foundation for systematically examining whether, how, and which posteriors are represented in neural circuits during complex decision-making tasks.
University of Oxford
Training curriculum influences sharing of representations between tasks
Learning multiple tasks requires choosing how to allocate representations between tasks. Separating representations protects knowledge from interference and enables multitasking, while sharing allows for efficient learning and generalization (Musslick et al. 2017). However, studying which features of the training regime drive neural networks towards task sharing or task separation is difficult because of their susceptibility to catastrophic forgetting and plasticity loss. In this study, we leverage recent advances in training linear networks successfully across tasks whose representations are mediated by neural task abstractions (NTA; Sandbrink*, Bauer*, Proca* et al. 2024) to investigate the influence of curriculum on task sharing. In a block-like curriculum in which different tasks are presented sequentially, the NTA system learns a decomposition of the tasks into different specialized modules that makes use of the shared structure. To study the dynamics of separation, we restrict individual modules to be shared or separated and compare their relative learning speed in a competitive paradigm. We find that shared representations are encouraged by interleaved training, while separated representations are facilitated by a block-like task structure in which tasks are trained sequentially. We formalize this relationship using the exact solutions to the learning dynamics of neural networks with multiple paths (Shi et al. 2022). Since in human and animals representational sharing is hypothesized to change dynamically over the course of training but the exact dynamics are yet-to-be-understood, these findings allow us to make novel predictions for future paradigms
UC Berkeley
Storing data structures in synapses: combining vector symbolic architectures with tensor product representations
Standard computing uses symbols and data structures to design and execute programs. However, it is unclear how distributed patterns in artificial or biological neural networks can represent symbols and compound data structures. To address this problem, two different models were proposed, but each model has downsides. Tensor product representations (TPRs) of compound data structures consist of the outer product tensor of the patterns representing the individual components. Thus, TPRs require a representation space that grows exponentially with the depth of the compound data structure. In vector-symbolic architectures (VSAs), a data structure is represented by a lossy compression of the corresponding tensor. The resulting representations are of fixed dimension, independent of depth. Such a fixed representation dimension is convenient, but the compression loss limits the complexity of data structures that can be processed. Here, we propose a combination of TPR and VSA that uses flexible VSA-type neural representations to form and query TPR-type representations of data structures held in synaptic memory. This hybrid of VSA and TPR allows for both the dimensionality-preserving representation of deep and complex data structures through "pointers" described by neural activity, as well as high memory capacity through storage in synaptic weights using short-term synaptic plasticity. We demonstrate several examples of how to store and compute on these synaptic data structures, such as analogical reasoning, tree search, and visually guided navigation. Finally, we derive a theory for the memory capacity and scaling of hybrid VSA-TPR networks.
Carnegie Mellon University
Neural Networks Reveal a Cognitive Continuum Toward Human Abstraction
Neural networks struggle with symbolic abstraction, a core aspect of human cognition. Prior research primarily compares these models to U.S. adults, leaving it unclear whether they fundamentally diverge from human cognition or align with earlier developmental and evolutionary stages. We evaluate over 200 neural networks alongside U.S. adults, children, Tsimane adults, and macaques on three visual match-to-sample tasks probing different levels of abstraction. Specifically, we examine three key signatures of human cognition: (1) the semantic distance effect (semantic similarity influencing visual judgments), (2) the regularity effect (advantage for regular over irregular shapes), and (3) relational bias(preference for relational reasoning).
As abstraction increases, neural network decisions deviate further from human responses. Likewise, accuracy follows a continuum from macaques to children, Tsimane adults, and U.S. adults. Representational similarity analyses reveal significant correlations between model embeddings and the choice efficiency of reference groups, even when models diverge from human-like decisions, suggesting shared cognitive properties. We also explore how inductive biases—such as model architecture, size, training objectives, dataset composition, and language supervision—shape alignment with human cognition. While increased model depth sometimes weakens alignment with human semantic judgments, larger models, richer datasets, and language supervision enhance sensitivity to shape regularity.
This study uniquely integrates large-scale AI model comparisons with cross-species and cross-cultural evaluations. Our findings suggest that neural networks align along a cognitive continuum toward adult human abstraction, shedding light on the origins of abstract representation. These insights advance cognitive science, inform AI design, and bridge artificial and human cognition.
Forward College Berlin
Collective Moral Reasoning in Multi-Agent LLMs
Moral decisions are integral to aligning large language models (LLMs) to human values. As multi-agent systems gain prominence, it becomes crucial to understand how LLMs function collectively during collaboration, compared to individual LLMs. In human moral decisions, group deliberation leads to a utilitarian boost: a tendency to endorse norm violations that maximize benefits for the greatest number, despite harms. We study whether a similar dynamic emerges in multi-agent LLM systems. We tested six models on well-established sets of moral scenarios across two conditions: (1) Solo, where models reasoned independently, and (2) Group, where they engaged in multi-turn discussions in pairs or triads. In personal scenarios where agents must decide to directly harm one to save many, all models found moral violations to be more acceptable when part of a group than individuall. Some models endorsed actions that maximized overall well-being, even if they benefited strangers over familiar individuals. Others became more willing to violate moral norms in groups. While human groups show a similar group bias, the mechanism for their utilitarian boost differs from LLMs. This suggests that while the surface behavior of LLM collectives mimics human group reasoning, the underlying drivers differ. We discuss the implications for AI alignment.