Artificial Intelligence
Machine learning and artificial intelligence (AI) aim to create algorithms that solve difficult problems and simulate complex intelligent behavior. Many of these algorithms are based on findings and theory from the study of the brain and mind.
Recent rapid advances in these fields have seen the creation of algorithms and agents that can—finally—solve complex real-world problems across a wide range of domains. What are these advances, and how can we take them further? What remains beyond their capacity, and how can we overcome that? What might forever lie beyond their capabilities—or will anything?
Session Chairs
Michael Lepori (Brown University)
Dr Ivana Kajic (Google DeepMind)
Keynote Talks
Professor Jay McClelland (Director, Center for Mind, Brain, Computation and Technology, Stanford)
Professor Juergen Schmidhuber (Director, KAUST AI Initiative)
Invited Talks
Spotlight Talks
Can Demircan (Helmholtz Munich)
Professor Constantine Dovrolis (The Cyprus Institute / Georgia Tech)
Dr Venkatakrishnan Ramaswamy (Birla Institute of Technology & Science Pilani)
Andrea Albert (HUN-REN Wigner Research Centre for Physics)
Domonkos Martos (HUN-REN Wigner Research Centre for Physics)
Jack Brady (Max Planck Institute for Intelligent Systems)
Keynotes
Stanford and DeepMind
Director, KAUST AI Initiative
Are People Still Smarter than Machines?
Today, AI systems are being put to use for many purposes, and some systems have exceeded human capabilities. But are these systems really intelligent? There are good reasons to think that humans still have many advantages. In this talk I will discuss what I see as the advantages humans still have and suggest ways in which the AI systems of the future might capture them.
Falling Walls, WWW, Modern AI, and the Future of the Universe
Around 1990, the Berlin Wall came down, the WWW was born at CERN, mobile phones became popular, self-driving cars appeared in traffic, and modern AI based on very deep artificial neural networks emerged, including the principles behind the G, P, and T in ChatGPT. I place these events in the history of the universe since the Big Bang, and discuss what's next: not just AI behind the screen in the virtual world, but real AI for real robots in the real world, connected through a WWW of machines. Intelligent (but not necessarily super-intelligent) robots that can learn to operate the tools and machines operated by humans can also build (and repair when needed) more of their own kind. This will culminate in life-like, self-replicating and self-improving machine civilisations, which represent the ultimate form of upscaling, and will shape the long-term future of the entire cosmos. The wonderful short-term side effect is that our AI will continue to make people's lives longer, healthier and easier.
Invited Speakers
Google DeepMind
From Open-ended Experience to Wisdom: Where Memories in Brains and AIs Converge
Humans exhibit a lifelong capacity for transforming experience into profound wisdom, a trajectory current AI systems have yet to achieve; bridging this gap is therefore a critical research frontier. This presentation explores key neuroscientific insights, such as how the brain abstracts experiences into discrete pointers that are generalizable, and employs mechanisms like awake hippocampal replay to 'tag' memories for sparse consolidation. The translation of these principles, particularly sparsity, offers significant benefits to AI, demonstrated in reinforcement learning through 'Contrastive Retrospection' (ConSpec), which rapidly identifies critical states to enhance learning and generalization. Next, we pivot to current learning in Large Language Models, revealing how they learn and forget, and how newly ingested data reshapes their existing knowledge landscape in ways that differ substantially from the brain when confronting similar continual learning challenges. Finally, we discuss how inspiration from the brain is able to substantially improve on-target continued learning in LLMs and reduce hallucinations. By incorporating principles of the brain for long term memory and continued learning, we aim to endow AIs with the capacity to transform their own journeys into genuine, evolving wisdom.
Google DeepMind
Visual Number Sense in Generative AI Models
Recent years have seen a proliferation of AI models that are capable of producing high-quality images that faithfully depict concepts described using natural language. Such models can generate images that represent arbitrary objects, object attributes, and complex relations between objects. In this talk, I will show that despite these impressive advancements, such models can still struggle with relatively simple tasks. Specifically, I will demonstrate that even the most advanced models have only a rudimentary notion of number sense. Their ability to correctly generate a number of objects in an image is limited to small numbers, and it is highly dependent on the linguistic context the number term appears in. I will further highlight challenges associated with evaluation of different model capabilities, including evaluation of numerical reasoning, and talk about different automated approaches that can be used to evaluate models in a more interpretable way by leveraging existing tools in machine learning and cognitive science.
Harvard
You Know It Or You Don’t: Phase Transitions, Clustering, and Random Variation in Language Model Training
While years of scientific research on model training and scaling assume that learning is a gradual and continuous process, breakthroughs on specific capabilities have drawn wide attention. Why are breakthroughs so exciting? Because humans don’t naturally think in continuous gradients, but in discrete conceptual categories. If artificial language models naturally learn discrete conceptual categories, perhaps model understanding is within our grasp. In this talk, In this talk, I will describe what we know of categorical learning in language models, and how discrete concepts are identifiable through empirical training dynamics and through random variation between training runs. These concepts involve syntax learning, out of distribution generalization, and “emergent” capabilities. By leveraging categorical learning, we can ultimately understand a model's natural conceptual structure.
(Brown University)
More Than Meets the Eye: Revealing Hidden Similarities Between Models and Minds Using Mechanistic Interpretability
Language models (LMs) can perform a wide array of complex cognitive tasks, sometimes in a manner that is strikingly human-like. However, it is not at all clear that LMs accomplish these tasks using mechanisms and representations that are similar to those underlying human behavior. In this talk, I will discuss our ongoing work that employs techniques from mechanistic interpretability to characterize the relationship between LM internals and human cognition in two distinct domains. First, I analyze LM’s internal representations of modal concepts (such as possibility), and demonstrate that these internal representations explain human behavioral judgments of such concepts. Next, I show that both models and humans are subject to competitor interference effects when performing tasks like factual recall. This similarity allows us to predict human DVs related to processing difficulty from metrics characterizing LM processing. Finally, I will discuss the implications of this work for the relationship between LMs and cognitive science.
Spotlight Talks
Helmholtz Munich
Sparse Autoencoders Reveal Temporal Difference Learning in Large Language Models
In-context learning, the ability to adapt based on a few examples in the input prompt, is a ubiquitous feature of large language models (LLMs). However, as LLMs' in-context learning abilities continue to improve, understanding this phenomenon mechanistically becomes increasingly important. In particular, it is not well-understood how LLMs learn to solve specific classes of problems, such as reinforcement learning (RL) problems, in-context. Through three different tasks, we first show that Llama 70B can solve simple RL problems in-context. We then analyze the residual stream of Llama using Sparse Autoencoders (SAEs) and find representations that closely match temporal difference (TD) errors. Notably, these representations emerge despite the model only being trained to predict the next token. We verify that these representations are indeed causally involved in the computation of TD errors and Q-values by performing carefully designed interventions on them. Taken together, our work establishes a methodology for studying and manipulating in-context learning with SAEs, paving the way for a more mechanistic understanding.
Professor Constantine Dovrolis
(The Cyprus Institute / Georgia Tech)
Hierarchically Modularity in Neural Networks Improves Generalization and Learning Efficiency
In both natural and artificial intelligence, many learning tasks have an underlying hierarchical and modular structure, composed of smaller sub-functions. Traditional artificial neural networks (NNs) often disregard this structure, leading to inefficiencies in learning and generalization. Prior work has demonstrated that leveraging known structural information can enhance performance by aligning NN architectures with the task’s inherent modularity. However, the extent of prior structural knowledge required to achieve these performance improvements remains unclear. In this work, we investigate how modular NNs can outperform traditional dense NNs on tasks with simple yet known modular structure by systematically varying the degree of structural knowledge incorporated. We compare architectures ranging from monolithic dense NNs, which assume no prior knowledge, to hierarchically modular NNs with shared modules that leverage sparsity, modularity, and module reusability. Our experiments demonstrate that module reuse in modular NNs significantly improves learning efficiency and generalization. Furthermore, we find that module reuse enables modular NNs to excel in data-scarce scenarios by promoting functional specialization within modules and reducing redundancy.
Birla Institute of Technology & Science Pilani
Memorizing Deep Networks can Generalize (much better)
Overparameterized Deep Networks that generalize well have been key to the dramatic success of Deep Learning. The reasons for their remarkable ability to generalize are not well understood yet. It has also been known that Deep Networks possess the ability to memorize training data, as evidenced by perfect or high training accuracies on models trained with corrupted data that have class labels shuffled to varying degrees. Concomitantly, such models suffer from poor test accuracies, due to which it is thought that the act of memorizing substantially degrades the ability to generalize. It has, however, been unclear why the poor generalization that accompanies such memorization, comes about. One possibility is that during training, the layers of the network irretrievably re-organize their representations, making generalization difficult. The other possibility is that the network retains significant ability to generalize, but the trained network somehow “chooses” to readout in a manner that is detrimental to generalization. Here, we provide evidence for the latter possibility by demonstrating, empirically, that such models possess information in their representations for substantially improved generalization. Furthermore, such generalization abilities can be easily decoded from the internals of the trained model, and we build a technique to do so. We demonstrate results on multiple models trained with standard datasets. The results here are reminiscent of a puzzling phenomenon in Neuroscience wherein it has been shown that a decoder using data from a subset of neurons from a well-trained animal can have accuracy significantly better than the behavioral accuracy of the animal.
HUN-REN Wigner Research Centre for Physics
Orthogonal task representations prevent catastrophic forgetting in continual learning
A central challenge for continual learning is catastrophic forgetting, where acquiring new knowledge overwrites previously learned information causing the model to forget earlier tasks. Here, we investigate this problem from a representational geometry perspective using recurrent neural networks (RNNs). Specifically, we examine task representations, the internal encodings of task-relevant information in the network's activity space, and analyze how their structure and alignment influence forgetting. While various approaches explicitly mitigate catastrophic forgetting, such as elastic weight consolidation, memory replay, and so on, the spontaneous evolution of neural task representations facilitating continual learning without added mechanisms has received relatively little attention. Previous studies show that during gradient updates enforcing strict orthogonality between task representations eliminates interference. Building on this result, we hypothesize that in general more orthogonal task representations should lead to less forgetting. We test this hypothesis across a variety of stimulus and goal statistics, demonstrating how the degree of representational overlap shapes forgetting dynamics. Indeed, we found that greater representational overlap leads to more severe forgetting, whereas nearly orthogonal representations reduce forgetting. What ultimately appears to count, is the balance between forgetting and retaining previous tasks: Over multiple task shifts the network learns each task while gradually orthogonalizing its representations. It is worth noting that in tasks with shared goals, some residual representational overlap can persist without causing forgetting, reflecting an underlying shared structure between tasks. These findings highlight the importance of representational geometry in continual learning and demonstrate that well-factorized task representations alone do mitigate forgetting in neural networks.
HUN-REN Wigner Research Centre for Physics
Uncertainty in latent representations of variational autoencoders optimized for visual tasks
Optimal computations under uncertainty, both in natural and artificial intelligent systems, require an adequate probabilistic representation about beliefs, which can be captured in terms of latent variable models. Deep generative models (DGMs) have the potential to learn flexible models of data while avoiding intractable computations, pervasive in Bayesian inference. In particular, Variational Autoencoders (VAEs) promise to support optimal computations by establishing a principled basis for approximate probabilistic inference. However, we identify conditions under which VAEs systematically fail to provide accurate estimates of uncertainty, which affects multiple computer vision domains. We draw inspiration from classical computer vision to introduce an extension to the decoder of VAEs, the neural network that shapes learning. We demonstrate that the resulting Explaining-Away VAE (EA-VAE) remedies defective inference in VAEs. In particular, we show that, in contrast with standard VAEs, uncertainty estimates in the EA-VAE are consistent with normative requirements across a wide spectrum of perceptual tasks, including image corruption, interpolation, and out-of-distribution detection. Investigating the mechanisms that deliver restored inference capabilities, we discover that the encoder of the VAE (responsible for supporting making inferences) displays features of divisive normalization: a motif widespread in biological neural networks, which has been shown to contribute to favorable computational properties of the nervous system. Our results establish EA-VAEs as reliable tools to perform inference in deep generative models and highlight the contribution of neurally inspired computations to more precise inference.
Max Planck Institute for Intelligent Systems
Generation is Required for Robust Perception
It has been hypothesized and supported empirically that visual perception in mammalian brains leverages an internal generative model of the world. However, the most successful vision models in machine learning today are discriminative, relying only on an “encoder”, without a generative “decoder”. This raises the question if generation is necessary for the robust perception characteristic of natural intelligence.
In the present work, we provide a possible answer to this question by studying mathematically what is required for compositional generalization, a hallmark of the robustness of mammalian perception. In this setting, an agent must generalize to out-of-domain (OOD) scenes consisting of novel combinations of known concepts, e.g. objects. Under mild assumptions, we prove that compositional generalization is, in fact, theoretically impossible using a purely discriminative, encoder-only, model and, instead, requires learning a generative decoder.
Specifically, we prove that, for a broad family of data generating processes, it is not possible to formulate inductive biases on an encoder that enable compositional generalization. On the other hand, it is theoretically and empirically possible to impose inductive biases on a decoder such that it can correctly generate OOD scenes containing novel combinations of concepts. Taken together, these results imply a hybrid approach is necessary for compositional generalization where a decoder is used in tandem with an encoder. We show that this is achieved either by training an encoder offline on OOD generations from the decoder (replay), or online, by iteratively adapting an encoder’s representation of OOD scenes using decoder feedback (search). Notably, while replay and search emerge under our theory as necessary computations for compositional generalization, evidence of these processes has recently been reported in the hippocampal formation. Finally, we corroborate our theoretical results empirically on toy image data.