Jump to ContentJump to Main Navigation
Cognitive SearchEvolution, Algorithms, and the Brain$
Users without a subscription are not able to see the full content.

Peter M. Todd, Thomas T. Hills, and Trevor W. Robbins

Print publication date: 2012

Print ISBN-13: 9780262018098

Published to MIT Press Scholarship Online: May 2016

DOI: 10.7551/mitpress/9780262018098.001.0001

Show Summary Details
Page of

PRINTED FROM MIT PRESS SCHOLARSHIP ONLINE (www.mitpress.universitypressscholarship.com). (c) Copyright The MIT Press, 2022. All Rights Reserved. An individual user may print out a PDF of a single chapter of a monograph in MITSO for personal use.date: 24 June 2022

Model-Based Reinforcement Learning as Cognitive Search

Model-Based Reinforcement Learning as Cognitive Search

Neurocomputational Theories

(p.195) 12 Model-Based Reinforcement Learning as Cognitive Search
Cognitive Search

Nathaniel D. Daw

The MIT Press

One oft-envisioned function of search is planning actions (e.g., by exploring routes through a cognitive map). Yet, among the most prominent and quantitatively successful neuroscentific theories of the brain’s systems for action choice is the temporal-difference account of the phasic dopamine response. Surprisingly, this theory envisions that action sequences are learned without any search at all, but instead wholly through a process of reinforcement and chaining. This chapter considers recent proposals that a related family of algorithms, called model-based reinforcement learning, may provide a similarly quantitative account for action choice by cognitive search. It reviews behavioral phenomena demonstrating the insufficiency of temporal-difference-like mechanisms alone, then details the many questions that arise in considering how model-based action valuation might be implemented in the brain and in what respects it differs from other ideas about search for planning.

Keywords:   Strüngmann Forum Reports, cognitive search, planning actions, model-based reinforcement learning, decision algorithms, chaining, model-based valuation, phasic dopamine response, planning actions

MIT Press Scholarship Online requires a subscription or purchase to access the full text of books within the service. Public users can however freely search the site and view the abstracts and keywords for each book and chapter.

Please, subscribe or login to access full text content.

If you think you should have access to this title, please contact your librarian.

To troubleshoot, please check our FAQs, and if you can't find the answer there, please contact us.