Probabilistic Models

Design by Amey Zhang

How should an intelligent agent behave in order to best realize their goals? What inferences or actions should they make in order to solve an important computational task? Probabilistic models aim to answer these questions at an abstract computational level, using tools from probability theory and statistical inference.

In this session we will discuss how such optimal behavior should change under different conditions of uncertainty, background knowledge, multiple agents, or constraints on resource. This can be used to understand human behavior in the real world or the lab, as well as build artificial agents that learn robust and generalizable world models from small amounts of data.

Session Chair

Dr Ruairidh Battleday (Oxford University)

Probabilistic Models of Cognition and Machine Learning: past and future directions

Invited Talks

Professor Bill Thompson (University of California, Berkeley)

Distributed Computation by Social Learning

Contributed Talks

Professor Volker Tresp (Munich Center for Machine Learning)

The Tensor Brain: A Unified Theory of Perception, Memory and Semantic Decoding

Professor Daniel Graham (HWS)

Collision Models of Brain Network Communication

Rahul Jain (Pomona College)

You Got Hexxed: Persistence during Complex Skill Learning

Invited Talks

University of California, Berkeley

Distributed Computation by Social Learning

Much of what we know and how we behave is learned socially — by observing other people and listening to what they say. The transmission of representations and algorithms from person to person and across generations results in an evolutionary process, yet the dynamics of this process have traditionally been difficult study. I will describe a theory of human cognition that construes the transmission of cognitive algorithms from person to person as a form of distributed computation for efficient programs. I will present the results of a large-scale behavioral experiment in which thousands of participants attempted to solve a sequential decision-making task and pass on algorithmic solutions. I will discuss the algorithms people discovered and how they were shaped by transmission and selection at the population level.

Contributed Talks

Munich Center for Machine Learning

The Tensor Brain: A Unified Theory of Perception, Memory and Semantic Decoding

We present a unified computational theory of an agent’s perception and memory. In our model, both perception and memory are realized by different operational modes of the oscillating interactions between a symbolic index layer and a subsymbolic representation layer. The two layers form a bilayer tensor network (BTN). The symbolic index layer contains indices for concepts, predicates, and episodic instances known to the agent. The index layer labels the activation pattern in the representation layer and then feeds back the embedding of that label to the representation layer. The embedding vectors are implemented as connection weights linking both layers. An index is a focal point of activity and competes with other indices, but, since it constantly interacts with the representation layer, it is never active in isolation. Embeddings have an integrative character: the embedding vector for a concept index integrates all that is known about that concept, and the embedding vector for an episodic index represents the world state at that instance. The subsymbolic representation layer is the main communication platform. In cognitive neuroscience, it would correspond to, what authors call, the ``mental canvas'' or the ``global workspace'' and reflects the cognitive brain state. In bottom-up mode, scene inputs activate the representation layer, which then activates the index layer. In top-down mode, an index activates the representation layer, which might subsequently activate even earlier processing layers. Joint work with: Hang Li, Sahand Sharifzadeh, Hang Li, Dario Konopatzki, Yunpu Ma

Hobart and William Smith Colleges

Collision Models of Brain Network Communication

Coding models dominate neurotheory, yet with few exceptions, models of how neuronal networks manage dynamic message passing (routing) among nodes are lacking. Indeed, brains must both compute (encode/decode) and route (pass messages reliably, flexibly, and over distance) using the same highly interconnected neuronal network, and do so in the absence of global knowledge or control of network conditions and structure. A fundamental concern of all routing systems is message-message interactions but because they can generate nonlinearities, such interactions are difficult to study analytically. We offer evidence of emergent effects of one type of message interaction, destructive collisions, using numerical simulations of synchronous Markovian agents on generic networks and empirical mammal connectomes. We measured message flow under two routing policies: unbiased random walks (RW); and “information spreading” (IS), wherein messages arriving at a node without collision copy themselves to all first neighbors. In randomized undirected graphs sharing the same scale-free degree sequence, we find that network assortativity (similarity of node degree to average first-neighbor degree) regulates message lifetimes in fundamentally different ways for RW and IS. Average message survival time increases monotonically from highly disassortative to highly assortative graphs under RW, while under IS average lifetime has a broad peak at zero to moderate assortativity. We then show that directed mouse and monkey connectomes, which are slightly to moderately disassortative, behave similarly to generic networks of the corresponding assortativity under RW. We conclude that message interactions play a fundamental role in shaping network dynamics and that neural coding models must ultimately take account of such dynamics.

Rahul Jain

Pomona College

You Got Hexxed: Persistence during Complex Skill Learning

In the last decade, AIs trained using reinforcement learning (RL) have reached human-level performance on many challenging games. These AIs, however, require extensive training as they struggle to draw from prior knowledge and generalize from limited experience. Modeling human learning is difficult because a) many lab-based tasks are too constrained to distinguish human learning from simple RL models, and b) “real-world” games are too unconstrained to sample the learning process parametrically. To fill this gap, we designed hexxed (https://hexxed.io), a medium-complex game-based task in which we can observe an individual’s actions as they learn to solve a puzzle—expressible as a deterministic MDP—over many attempts. We have collected data from >10k online subjects, helping us observe regularities in how people arrive at solutions. As a “null model” for learning, we trained a three-layer convolutional Deep-Q Network (DQN), a model-free RL agent. We observe three significant traits that distinguish humans from DQNs: • People are “picky”: Subjects opt for target-centered actions, carving a strategy "highway" in the MDP; DQNs uniformly sample MDP. • People are “sticky”: Unlike DQNs, which continually update their policies, humans sample the same strategy repeatedly despite being unrewarding. • People have “leaps of insight”: Unlike gradual DQN learning, humans often transition suddenly after a reward drought to a highly rewarding, often optimal, policy. Observing individual players' learning trajectories suggests they are foraging over strategy space. Strategies can be seen as choices within an N-armed bandit problem in which subjects must balance exploring new strategies and exploiting known ones to maximize reward.