dp.quantecon.org

Dynamic Programming

Thomas J. Sargent · John Stachurski

These books offer a unified treatment of dynamic programming, from finite-state foundations to a complete reworking of the theory for general state spaces. Applications span economics, finance, and operations research.

Read & download

Online editions are searchable and include runnable code. PDFs remain available for offline use.

Overview Lectures

High-level introduction covering both volumes.

Lecture 1 (PDF) Lecture 2 (PDF)

Contents

Volume I — Finite States

  1. Introduction
  2. Operators and Fixed Points
  3. Markov Dynamics
  4. Optimal Stopping
  5. Markov Decision Processes
  6. Stochastic Discounting
  7. Valuation
  8. Recursive Decision Processes
  9. Abstract Dynamic Programming
  10. Continuous Time

Volume II — General States

  1. Prelude: Examples of Dynamic Programs
  2. Abstract Decision Processes
  3. ADPs on Pospace
  4. ADPs on Banach Space
  5. ADP Transformations
  6. Linear Decision Processes
  7. Recursive Decision Processes
  8. Additional Applications
  9. Approximation and Learning