Artificial Intelligence
Machine learning and artificial intelligence (AI) aim to create algorithms that solve difficult problems and simulate complex intelligent behavior. Many of these algorithms are based on findings and theory from the study of the brain and mind.
Recent rapid advances in these fields have seen the creation of algorithms and agents that can—finally—solve complex real-world problems across a wide range of domains. What are these advances, and how can we take them further? What remains beyond their capacity, and how can we overcome that? What might forever lie beyond their capabilities—or will anything?
In this session we will hear from some of the world’s leading experts in academia and tech. We will also hear from proponents of structure, and from proponents of scale. And we will also hear some radical suggestions for reframing many fundamental problems of intelligence.
Keynote Talks
Dr Feryal Behbahani (Google DeepMind)
Professor Kevin Ellis (Cornell): Doing experiments and acquiring concepts using language and code
Session Chairs
Dr Ishita Dasgupta (Google DeepMind)
Dr Ilia Sucholutsky (Princeton University)
Invited Talks
Professor Najoung Kim (BU, Google): Comparing human and machine inductive biases for compositional linguistic generalization using semantic parsing: Results and methodological challenges
Professor Rafal Bogacz (Oxford): Modelling diverse learning tasks with predictive coding
Dr André Barreto (DeepMind)
Dr Wilka Carvalho (Harvard): Predictive representations: building blocks of intelligence
Spotlight Talks
Declan Campbell (Princeton): Decomposing Visual Reasoning: A Study of Sequential Task Processing in Multimodal Language Models
Quentin Ferry (MIT): Emergence and Function of Abstract Representations in Self-Supervised Transformers
Shubham Choudhary (Harvard): Implicit generative models using kernel similarity matching
Michael Spratling (University of Luxembourg): A margin-based replacement for cross-entropy loss that improves the robustness of deep neural networks on image classification tasks
Luke Eilers (University of Bern): A generalized neural tangent kernel for surrogate gradient learning
Samuel Lippl (Columbia University): The impact of task structure, representational geometry, and learning mechanism on compositional generalization
Anita Keshmirian (Ludwig Maximilian University of Munich): Investigating Causal Judgments in Humans and Large Language Models
Sunayana Rane (Princeton): Can Generative Multimodal Models Count to Ten?
Michael Lepori (Brown): A Mechanistic Analysis of Same-Different Relations in ViTs
Claudius Gros (Goethe University Frankfurt): Neural scaling laws in AlphaZero
Paul Riechers (Beyond Institute for Theoretical Science; BITS): Computational mechanics predicts internal representations of transformers
Felix Binder (UCSD): Lessons learned in the study of representational alignment in physical reasoning
Cornell University
Doing experiments and acquiring concepts using language and code
Human inductive learning is rapid: From relatively few examples, we can learn the rules of a new game or the norms of a new culture. Inductive learning is also broad: the space of learnable concepts is effectively unbounded, because simpler concepts can compose to build bigger ones. In this talk I propose an inductive learning model whose aim is to be more humanlike in that it is broader-coverage, while supporting learning that is rapid both in the number of examples and the amount of compute required. The model combines language models with Bayesian reasoning and neural code generation. I will also discuss an extension of the model which performs active learning, where the model proposes experiments that best triangulate the target concept. The resulting model approaches human performance on three concept-learning setups, and gives a reasonably close fit to human behavioral data. Together these results suggest an architecture for more human-like inductive learners, which can both learn from examples and also propose basic experiments and ask questions, and which is organized around the approach of equipping language models with explicitly Bayesian machinery.
Joint work with students Top Piriyakulkij, Cassidy Langenfeld
Invited Talks
Boston University/ Google
Comparing human and machine inductive biases for compositional linguistic generalization using semantic parsing: Results and methodological challenges
TBA
University of Oxford
Modelling diverse learning tasks with predictive coding
Predictive coding (Rao & Ballard, 1999) is an influential model describing information processing in hierarchically organized cortical circuits. It can learn effectively while only relying on local Hebbian plasticity. The predictive coding model was originally developed to describe unsupervised learning of representation in the visual cortex. This talk will give an overview of recent work extending predictive coding to diverse tasks, including: probabilistic inference, temporal prediction, supervised learning, memory, and novelty detection. The versatility of predictive coding supports that it is a promising model for a fundamental algorithm employed by cortical circuits.
Harvard University
Predictive representations: building blocks of intelligence
TBA
Spotlight talks
Princeton University
Decomposing Visual Reasoning: A Study of Sequential Task Processing in Multimodal Language Models
Empirical evidence from cognitive science indicates that humans tackle complex reasoning tasks by breaking them down into simpler, sequential sub-tasks. Inspired by this approach, recent advancements in machine learning have demonstrated that large language models (LLMs), such as GPT-4, benefit from analogous decomposition imposed by methods like chain-of-thought (CoT) and tree-of-thought (ToT). However, the necessity of sequential reasoning in multimodal contexts remains under-explored. Our study addresses this gap by evaluating GPT-4v on three visual reasoning tasks adapted from cognitive science: serial search, pop-out detection, and object counting. We find that while GPT-4v excels in the pop-out task, its performance deteriorates in serial search and counting as the number of objects increases—tasks where humans are known to employ sequential strategies. These findings highlight a limitation in current multi-modal models' ability to handle tasks requiring sequential reasoning, such as counting. Consequently, integrating serialization techniques inspired by CoT and ToT may be a promising direction for future research, potentially narrowing the gap between machine and human performance in visual reasoning.
Massachusetts Institute of Technology
Emergence and Function of Abstract Representations in Self-Supervised Transformers
Our brain's ability to form and exploit abstract world models allows us to rapidly navigate new situations, a trait that deep learning systems have historically struggled to replicate. Motivated by recent results showing that foundation models are few-shot learners, this study asks whether self-supervised transformers learn in silico abstract world models. We test this hypothesis by studying small-scale transformers trained to reconstruct partially masked visual scenes generated from a latent blueprint. We show that the networks develop linearly separable low-dimensional manifolds that encode all semantic features of the blueprint, hence forming an abstract model of the dataset. Using precise manipulation experiments, we demonstrate that these abstractions are central to the network's decision-making process. Additionally, we find that transformers organize abstractions in part-whole hierarchies that capture the compositional nature of the dataset. Finally, we introduce a novel architecture that grants us access to the learned abstractions, allowing us to more readily steer the network's decision-making process.
Harvard
Implicit generative models using kernel similarity matching
Understanding how the brain encodes representations for a given stimuli is a key question in neuroscience and has influenced the development of artificial neural networks with brain-like learning abilities. Recently, learning representations by capturing similarity between input samples has been studied to answer this question in the context of learning downstream features from the input. However, this approach has not been studied in the case of a generative paradigm, crucial for explaining top-down interactions in sensory processing, consistent with the predictive abilities of our neural circuitry. We propose a similarity matching framework for generative modeling. We show that representation learning under this scheme can be achieved by maximizing similarity between the input kernel and a latent kernel which leads to an implicit generative model arising from learning the kernel structure in the latent space. We argue that the framework can be used to learn input manifold structures, potentially giving insights into task representations in the brain. Finally, we suggest a neurally plausible architecture to learn the model parameters linking representation learning using similarity matching with predictive coding.
University of Luxembourg
A margin-based replacement for cross-entropy loss that improves the robustness of deep neural networks on image classification tasks
CE loss is standard for training DNNs to perform classification. However, there are many sub-tasks within the domain of classification where CE is sub-optimal. To address this issue we propose high-error margin (HEM) loss. Due to being a margin loss, and in contrast to CE loss, HEM stops modifying the weights once the activation of the target logit sufficiently exceeds the values of the other logits. This helps prevent the trained network making over-confident predictions and reduces over-fitting. Experiments show that HEM is as, or more, effective than CE across a wide range of tasks: unknown class rejection, adversarial robustness, learning with imbalanced data, continual learning, and semantic segmentation. The sub-optimal performance of CE has previously inspired several specialised losses. For example, LogitNorm (LN) trains networks better able to identify, and reject, samples from unknown classes. Logit adjusted (LA) loss improves performance when training data contains imbalanced numbers of samples in each category. For semantic segmentation DICE is one specialised loss among many. Our results show that HEM often out-performs these specialised losses, and in contrast to them, is a general-purpose replacement for CE loss.
University of Bern, Department of Physiology
A generalized neural tangent kernel for surrogate gradient learning
State-of-the-art neural network training methods depend on the gradient of the network function. For networks with activation functions without well-defined derivatives, such as spiking neural networks or binary neural networks, these highly successful methods therefore cannot be applied directly. To overcome this problem, the activation function's derivative is substituted with a surrogate derivative, giving rise to surrogate gradient learning, also known as straight-through estimation. As this method works well in practice, the development of a better theoretical understanding is desirable. In this work, we consider randomly initialized networks in the infinite-width limit using the neural tangent kernel (NTK). We study an extension of the NTK to activation functions with jumps and generalize the NTK to gradient descent with surrogate derivatives, i.e., surrogate gradient learning. This generalization includes the key theorems on the NTK, providing a mathematical foundation for the analysis of surrogate gradient learning in the infinite-width limit. Finally, we link our theoretical analysis to numerical investigations in the literature and conduct numerical experiments to illustrate our results.
Columbia University
The impact of task structure, representational geometry, and learning mechanism on compositional generalization
Compositional generalization (the ability to respond correctly to novel arrangements of familiar components) is thought to be a cornerstone of intelligent behavior. However, a theory of how and why models generalize compositionally across diverse tasks remains lacking. To make progress on this topic, we consider compositional generalization for kernel models with fixed, potentially nonlinear representations and a trained linear readout. We prove that they are limited to conjunction-wise additive compositional computations, and identify compositionality failure modes that arise from the data and model structure. For models in the representation learning (or rich) regime, we show that networks can generalize on an important non-additive task (transitive equivalence) and give a mechanistic explanation for why. Finally, we validate our theory empirically, showing that it captures the behavior of a convolutional neural network trained on a set of compositional tasks. In sum, our theory characterizes the principles giving rise to compositional generalization in kernel models, shows how representation learning can overcome their limitations, and provides a taxonomy of compositional tasks that may be useful beyond the models studied here.
Ludwig Maximilian University of Munich
Investigating Causal Judgments in Humans and Large Language Models
This study investigates biases in causal reasoning among humans and Large Language Models (LLMs) using Causal Bayesian Networks (CBNs), focusing on Canonical Chain (A→B→C) and Common Cause (A←B→C) structures. In these structures, once the intermediate variable (B) is known, the probability of the outcome (C) is normatively independent of the initial cause (A). However, studies have shown that humans often ignore this independence. We tested the mutually exclusive predictions of three theories that could account for this bias (N=300). We found that humans tend to perceive causes in Chain structures as significantly more powerful, providing support for only one of the hypotheses. We then examined if LLMs—trained on language data, including GPT3.5-Turbo, GPT4, and Luminous Supreme Control—exhibit similar biases by adjusting a key 'temperature' hyperparameter. By computing Earthmover’s distance, findings reveal that LLMs, especially at higher temperatures, display a comparable inclination towards Chain structures, suggesting this bias partly arises from language use. These results have significant implications for understanding causal reasoning in humans and Large Language Models.
Princeton University
Can Generative Multimodal Models Count to Ten?
We adapt a developmental psychology paradigm to characterize the counting ability of the foundation model Parti. The Give-N task is often used as the gold-standard for measuring a child's understanding of number concepts. Generative vision and language models can now be probed using something similar to the Give-N task, prompted with text like "five lemons" and asked to generate an image from scratch. We adapt the Give-N task to show that three model scales of the Parti model (350m, 3B, and 20B parameters respectively) each have some counting ability, with a significant jump in performance between the 350m and 3B model scales. We also demonstrate that it is possible to interfere with these models' counting ability simply by incorporating unusual descriptive adjectives for the objects being counted into the text prompt (such as "one hairy orange" -- see Figure 1). We analyze our results in the context of the knower-level theory of child number learning, and illustrate the corresponding gaps in model learning. Notably, we find that the performance boost children gain once they understand the inductive step of counting (that each subsequent number can be counted by adding one to the previous number) is missing from all three scales of model behavior. Our results show that we can gain experimental intuition for how to probe model behavior by drawing from a rich literature of behavioral experiments on humans, and, perhaps most importantly, by adapting human developmental benchmarking paradigms to AI models, we can characterize and understand their behavior with respect to our own.
Brown University
A Mechanistic Analysis of Same-Different Relations in ViTs
Vision transformers (ViTs) have achieved state of the art performance on a variety of tasks, yet little is known about the algorithms they learn to solve these tasks. To investigate this, we adopt techniques from mechanistic interpretability to uncover how ViTs perform a simple abstract visual reasoning task: judging whether two objects are the same or different. Even when models achieve similar test accuracy, we reveal that pretrained ViTs adopt qualitatively different algorithms than ViTs trained from scratch on this task. Specifically, the pretrained models’ strategy aligns with the algorithms implemented by computational models of human visual reasoning. Additionally, we reveal that the representations learned by pretrained ViTs are mathematically well-structured, separating distinct visual features (i.e. shape and texture) into separate linear subspaces. This enables precise control over model behavior with surgical interventions. Finally, we relate the linear structure of features in hidden representations to generalization behavior. Our work provides a case study for applying mechanistic interpretability techniques to ViTs while also providing insights into the algorithmic and representational benefits of pretraining.
Goethe University Frankfurt
Neural scaling laws in AlphaZero
Large language model (LLM) training relies on scaling laws, the predictable scaling of performance and optimal model size as a power law of training compute. These power laws appear in many supervised learning settings, such as language and vision, but are mostly absent from reinforcement learning (RL). We present the first extensive analysis of performance power-law scaling of an RL algorithm, AlphaZero. Training a suite of agents, we find that performance on two popular board games follows power laws in both model size and training compute, with universal exponents. We show that the optimal model size scales as a power of the compute budget. This scaling law allows optimal training of large, expensive models, similar to LLM scaling laws. We also look for the origin of AlphaZero’s scaling laws. Following a theory explaining LLM scaling, we find a possible connection between performance power-law scaling and Zipf’s law. The frequency of game states generated by our agents during training follows a power law in rank, affecting agent training. We discuss the origin of Zipf’s law, and show that agents focus on optimizing the most frequent states first, in agreement with LLM scaling law theory.
Beyond Institute for Theoretical Science (BITS)
Computational mechanics predicts internal representations of transformers
Computational mechanics studies the limits of prediction: How much can you predict? What kind of structure is required for optimal prediction? These questions are relevant to both anticipating and interpreting advanced AI systems. Adapting the mathematical framework of computational mechanics, we have been able to predict both (i) internal representations and (ii) the precise decay of next-token entropy as a function of context position, for transformers pre-trained as usual to minimize next-token-prediction loss. We train small transformers across a variety of increasingly complex correlated stochastic processes and verify that (i) activations in the residual stream and (ii) in-context learning indeed both conform to our predictions.
UCSD
Lessons learned in the study of representational alignment in physical reasoning
Recent developments allow AI systems to perform cognitively complex and rich tasks. At the same time, collecting human behavior at scale is more feasible than ever. This convergence of trends allows for the combined large-scale study of human and AI behavior in rich domains and tasks. Such experiments promise to provide better insight into the representations and strategies underlying both human and AI behavior. However, doing so in a way that does justice to both humans and AI systems is challenging. Here, we outline the key considerations and challenges we've faced in a benchmarking study investigating physical understanding across humans and AI systems and discuss how we've addressed them.