Dynamic Programming and Optimal Control

Description
Dynamic Programming Algorithm; Infinite Horizon Problems; Value/Policy Iteration; Deterministic Systems and Shortest Path Problems; Deterministic Continuous-Time Optimal Control.

Reading Material
Dynamic Programming and Optimal Control by Dimitri P. Bertsekas, Vol. I, 3rd edition, 2005, 558 pages.

Requirements
Knowledge of differential calculus, introductory probability theory, and linear algebra.

Exam
There is a written final exam during the examination session, which covers all material taught during the course, i.e. the material presented during the lectures and corresponding problem sets, programming exercises, and recitations.

Grading
The grade will be given based on the final exam only.

Repetition
The performance assessment is only offered in the session after the course unit. Repetition only possible after re-enrolling.

Lectures

The lectures will be held in person in HG F1. The recordings can be found in the Video Portal. 

Recitations

The recitations will be held in person in HG F1. The recordings can be found in the Video Portal. The teaching assistants will cover the material of the previous week. 

Piazza Forum

Students are encouraged to post questions regarding the lectures and problem sets on the Piazza forum. The access code will be announced via email before the first lecture. We welcome everyone to try to answer and discuss about the questions posted. 

Programming Exercise (optional)

During the semester, there will be a programming exercise, which requires the student to apply the course material. Though no bonus points will be given for the programming exercise, we strongly encourage the students to do it as it helps to better understand the content. A maximum of two students can work as one team. The programming exercise will be in Python. 

For the top three submissions, we will issue prizes including a tour of Verity with Prof. D'Andrea, signed certificates and gift vouchers of 100 CHF.

Results: PE_results.pdf

Problem Sets

We will make sets of problems and solutions available online for the chapters covered in the lecture. It is the student's responsibility to solve the problems and understand their solutions. The problem sets contain programming exercises that require the student to implement the lecture material Python.

For Python package management, we recommend you use external page Anaconda Distribution, and for general installation and usage of Anaconda, please refer to the external page Anaconda Documentation. We will use Jupyter Notebook for the Python solutions, and for the general installation and usage of Jupyter Notebook, please refer to the external page Documentation. You can also choose to upload the notebook files onto your external page Google Colab and run it. If you have any problems running the provided solutions, please feel free to reach us.