The ability to flexibly adjust behavior in response to environmental changes is crucial for survival. This symposium will explore psychophysiological and computational mechanisms underlying expectation adaptation in reinforcement learning, integrating perspectives from both reward and threat learning. By leveraging eye-tracking, psychophysiology (e.g., EDA, ECG, pupil dilation, fMRI), virtual reality, and computational modeling, we aim to elucidate how individuals dynamically update their expectations in changing environments and adjust their behavior accordingly.

The session will begin with Ecem Tavacioglu (University of Würzburg), who will present findings on whether attentional biases during fear acquisition and reversal learning are primarily driven by threat value, predictability, or uncertainty. Next, Ondrej Zika (Karolinska Institute & MPIB) will examine how trait anxiety influences the mechanisms of context-dependent learning in changing aversive environments. Building on these insights, Francesco Tortora (University of Würzburg) will explore how the intricate interplay between elemental and contextual representations in fear learning shapes physiology and behavior within a Virtual Reality (VR) environment. Shifting toward developmental and clinical perspectives, Lorenz Deserno (University Hospital Würzburg) will present research on how ADHD impacts on reversal learning and its neural correlates (fMRI) in childhood and adolescence and how this relates to psychostimulant treatment response. Finally, Florian Schlagenhauf (Charité – Universitätsmedizin Berlin) will focus on goal-directed control in individuals with alcohol use disorder by investigating how reinforcement learning mechanisms contribute to maladaptive decision-making in addiction.

This symposium will provide a comprehensive multimodal perspective on the psychophysiological foundations of flexible behavioral adaptation, bridging fundamental research with clinical implications.

Threatening stimuli rapidly draw and sustain visual attention, yet it remains unclear whether this bias reflects inherent threat properties or the predictability of outcomes. Additionally, attentional exploration is shaped by learning that tend to favor more reliable predictors of upcoming events. To investigate the interplay between attention and learning during fear acquisition and reversal, we implemented a novel multiple-cue paradigm. Throughout the experiment, participants were presented with both single-cue and randomly interspersed multiple-cue displays. The single-cues differed in how reliably they predicted an aversive electrotactile stimulus: reliably predictive threat and safety cues, and ambiguous cues that were followed by the aversive stimulus on %50 of trials. In multiple-cue trials, ambiguous cues were shown alongside either threat or safety cues to examine whether attentional exploration is guided more by threat value or by the predictive strength of individual cues. Cue–outcome associations were initially learned during the acquisition phase and then reversed in the second half of the experiment to assess the flexibility of expectation updating (reversal learning). We collected eye-tracking data (e.g., first fixation latency, dwell time), trial-wise shock expectancy ratings, and autonomic responses. Participants flexibly adjusted their expectations to new contingencies; nevertheless, continued to exhibit a threat-related bias consistent with a better-safe-than-sorry strategy. Interestingly, gaze patterns were more sensitive to cue uncertainty, indicating that visual exploration may be more influenced by the unpredictability of outcomes. Ongoing reinforcement learning modeling will further clarify the relative contributions of threat value and outcome predictability in shaping attention and learning under changing aversive environments.

Previous work has linked trait anxiety to increased context-dependent learning, a mechanism believed to underlie increased rates of return-of-fear in anxiety (Zika et al., 2023). However, whether this tendency reflects correct inference of complex contextual environments or whether it becomes maladaptive along the apathy-anxiety spectrum has not been investigated. Here, we develop a novel contextual inference paradigm in which we manipulate the number of truly relevant contexts and assess whether anxious individuals tend to under- or over-estimate relevance of contextual information. The preliminary results indicate that participants are on average able to infer the true number of contexts (mean err = 0.34). We then link individual differences in contextual inference to a number of transdiagnostic dimensions (intolerance of uncertainty, anxiety, depression). Our preliminary results (N=18, target N=150) indicate that trait anxiety (TA) is associated with increased overall accuracy, r(17)=0.34, p<.05. Furthermore, high anxiety is associated with the correct number of contexts while low anxiety is associated with over-estimation of relevant contexts, r(17)=-0.22, p<.05. These initial results suggest that trait anxiety impacts the way individuals infer contextual information and provide a novel insight into how anxious individuals might deal with uncertainty.

Flexibly adapting behavior in a constantly changing world is crucial for survival. Processing and integrating contextual information is pivotal for constructing a coherent understanding of the world, particularly in potentially dangerous situations. Animal research highlights the role of the interplay between an elemental and a conjunctive context representation (dual-process theory) in guiding behavior. Here, we examined how these representations interact to influence human defensive responses using a virtual reality-based fear conditioning paradigm. The virtual environment consisted of three distinctly furnished office settings, each containing the same two lamps (one yellow and one blue). First, participants explored the three offices (context encoding). Then, they were repeatedly teleported within the two. In each office, one lamp (e.g., yellow) was paired with an aversive stimulation, while the other lamp (e.g., blue) was not. Critically, contingencies were reversed across contexts (context-dependent acquisition). In the final phase (behavioral test), participants could freely re-explore the three offices. Subjective ratings, autonomic responses (skin conductance, heart rate and pupil dilation), movement trajectories and gaze behavior were recorded. Ratings, autonomic responses and gaze behavior showed successful context-dependent learning, indicating dominance of conjunctive representation. Slightly reduced responses in the office where contingencies were reversed however demonstrated an influence of elemental representation. Despite distinguishing the role of the contexts and the elements, participants adopted a “better safe than sorry” exploration strategy during the behavioral test keeping thus more distance from both lamps. These findings reveal how contextual representations dynamically shape human defensive states.

Charting the maturation of flexible adaptation to changing action-outcome contingencies through the lense of reinforcment learning is key for developmental neuroscience and adjacent fields like computational psychiatry. We utilized different versions of reversal learning to examine (1) typical age-dependent development, (2) influence of an ADHD diagnosis and (3) the impact of the commonly prescribed psychostimulant methylphenidate. Using a task to separate motivational context and feedback valence, adolescence showed enhanced novelty seeking and response shifting especially after negative feedback, which lead to poorer returns when reward contingencies were stable. Computationally, this was explained by reduced sensitivity to positive reinforcement. Activity in the medial frontopolar cortex reflecting choice probability was attenuated in adolescence. Adult and childhood ADHD patients performed worse than controls particularly in trials before reversals, when reward contingencies were stable. Computational modelling showed decreased sensitivity to any reinforcement. This was reflected in a diminished representation of choice probability in the left posterior parietal cortex in ADHD. Using a modified task including partial versus complete counterfactual feedback in a pharmacological challenge study, methylphenidate specifically strengthened the impact of previous outcomes on choice behavior only in the partial feedback condition. In line with this pharmacological manipulation, we find in longitudinal data on treatment response in childhood ADHD that similar behavioral and copmutational signatures change with treatment over time; however, this change was, in a preliminary sample, not differentiable from repetition effects in the control group. These studies highlight the potential to understand developmental and clinical trajectories through computational models of reinforcement learning.