Dopamine, reward learning, and active inference

Thomas H B FitzGerald, Raymond J Dolan, Karl Friston

Research output: Contribution to journalArticle

35 Citations (Scopus)
5 Downloads (Pure)

Abstract

Temporal difference learning models propose phasic dopamine signaling encodes reward prediction errors that drive learning. This is supported by studies where optogenetic stimulation of dopamine neurons can stand in lieu of actual reward. Nevertheless, a large body of data also shows that dopamine is not necessary for learning, and that dopamine depletion primarily affects task performance. We offer a resolution to this paradox based on an hypothesis that dopamine encodes the precision of beliefs about alternative actions, and thus controls the outcome-sensitivity of behavior. We extend an active inference scheme for solving Markov decision processes to include learning, and show that simulated dopamine dynamics strongly resemble those actually observed during instrumental conditioning. Furthermore, simulated dopamine depletion impairs performance but spares learning, while simulated excitation of dopamine neurons drives reward learning, through aberrant inference about outcome states. Our formal approach provides a novel and parsimonious reconciliation of apparently divergent experimental findings.

Original languageEnglish
Article number136
JournalFrontiers in Computational Neuroscience
Volume9
DOIs
Publication statusPublished - 4 Nov 2015

Keywords

  • reward
  • reward learning
  • variational inference
  • dopamine
  • active interference
  • instrumental conditioning
  • incentive salience
  • learning

Cite this