INF2D: 29: Markov Decision Processes

The last section presented an approach to rational decision making in an uncertain static environment.  This section presents an approach to rational decision making in an uncertain dynamic environment.  In other words, we are going to combine the utility function with a dynamic Bayesian Network.  The result is known as a Markov Decision Process, and it forms the basis of a lot of decision making models in AI.

Lecture Slides

Previous Year's "Notes" Version

Required Reading

R&N Section 17.1–17.3  or NIE Chapter (17) "Making Complex Decisions", Sections 1–3.

NOTE: The abbreviation R&N refers to:

“Artificial Intelligence: A Modern Approach” Third Edition, Russell R & Norvig P, Prentice Hall, 2010 (R&N).

The abbreviation NIE stands for the following edition of the same book:

“Artificial Intelligence: A Modern Approach” Third Edition, Pearson New International Edition, Russell R & Norvig P, Pearson, 2014.