Abstract: Complex behaviors are often driven by an internal model, which integrates sensory information over time and facilitates long-term planning to reach subjective goals. We interpret behavioral data by assuming an agent behaves rationally --- that
is, they take actions that optimize their subjective reward according to their understanding of the task and its relevant causal variables. We apply a new method, Inverse Rational Control (IRC), to learn an agent's internal model and reward function by maximizing
the likelihood of its measured sensory observations and actions. Technically, we define an animal's strategy as solving a Partially Observable Markov Decision Process (POMDP), and we invert this model to find the task and subjective costs that have maximum
likelihood. This is a generalization of both Inverse Reinforcement Learning and Inverse Optimal Control. Our mathematical formulation thereby extracts rational and interpretable thoughts of the agent from its behavior. We apply this method to behavioral data
from primates catching fireflies in virtual reality, and use it to understand properties of the mental model monkeys use to navigate by optic flow.
The thoughts imputed to the animal can then serve as latent targets for neural analyses. Using these targets, we provide a framework for interpreting the linked processes of encoding, recoding, and decoding of neural data in light of the rational model for behavior. We first demonstrate the merits of this approach on synthetic neural data during a foraging task. We then analyze real neural activity in primate prefrontal cortex (PFC) and posterior parietal cortex (PPC) to discover computations relevant to foraging tasks. In PFC, we find that reward dynamics are represented in a subspace of the high-dimensional population activity, and predict animal’s subsequent choice better than either the true experimental variables or the raw neural responses. In PPC, we find representations of latent navigation-relevant variables, and find that task manipulations alter the coupling between neurons, suggesting that these interactions reflect the mental model used to perform task-relevant computations. Overall, our approach may identify explainable structure in complex neural activity patterns. This framework lays a foundation for discovering how the brain chooses to act using dynamic beliefs about the uncertain world.
Chair: Máté Lengyel (CEU, Cambridge)
Zoom Meeting ID:
995 7581 0673
Passcode: 635375
****************************************
Please, be informed that video/photo recording might take place at the event and the edited version of the video material might be published to communicate or promote CEU PU's activities. Please, find our Privacy Notice
here.
Best regards,
|
Ildikó Varga
Department
of Cognitive Science H-1051 Budapest, Nádor street 15. FT 404. CENTRAL EUROPEAN UNIVERSITY Quellenstrasse 51 | A-1100 Vienna | Austria |
www.ceu.edu
|