Dynamic Programming and Optimal Control

Lecture Details

Semester
2025 Fall

Course Catalogue
151-0563-01L

Credits
4 credit points

Time
Start: 17.09.2025
End: 26.11.2025

Lecturer
Prof. Raffaello D'Andrea

Assistants

Lecture
In person, HG F1, 14:15 - 16:00 on Wednesdays

Recitation
In person, HG F1, 15 minutes after the end of the lectures. Note that sometimes lectures end earlier.

Office Hours
By appointment (please send an e-mail to )

Description
Dynamic Programming Algorithm; Infinite Horizon Problems; Value/Policy Iteration; Deterministic Systems and Shortest Path Problems; Deterministic Continuous-Time Optimal Control.

Reading Material
Dynamic Programming and Optimal Control by Dimitri P. Bertsekas, Vol. I, 3rd edition, 2005, 558 pages.

Requirements
Knowledge of differential calculus, introductory probability theory, and linear algebra.

Exam
There is a written final exam during the examination session, which covers all material taught during the course, i.e. the material presented during the lectures and corresponding problem sets, programming exercises, and recitations.

Grading
The grade will be given based on the final exam only.

Repetition
The performance assessment is only offered in the session after the course unit. Repetition only possible after re-enrolling.

Distance examination
No distance examination will be offered.

Lectures

The lectures will be held in person in HG F1. The recordings can be found in the Video Portal. 

Recitations

The recitations will be held in person in HG F1. The recordings can be found in the Video Portal. The teaching assistants will cover the material of the previous week. 

Piazza Forum

Students are encouraged to post questions regarding the lectures and problem sets on the Piazza forum. The access code will be announced via email before the first lecture. We welcome everyone to try to answer and discuss about the questions posted. 

Programming Exercise (optional)

During the semester, there will be a programming exercise, which requires the student to apply the course material. Though no bonus points will be given for the programming exercise, we strongly encourage the students to do it as it helps to better understand the content. A maximum of two students can work as one team. The programming exercise will be in Python. 

For the top three submissions, we will issue prizes including a tour of Verity with Prof. D'Andrea, signed certificates and gift vouchers of 100 CHF.

Problem Sets

We will make sets of problems and solutions available online for the chapters covered in the lecture. It is the student's responsibility to solve the problems and understand their solutions. The problem sets contain programming exercises that require the student to implement the lecture material in Python.

For Python package management, we recommend you use external page Anaconda Distribution, and for general installation and usage of Anaconda, please refer to the external page Anaconda Documentation. We will use Jupyter Notebook for the Python solutions, and for the general installation and usage of Jupyter Notebook, please refer to the external page Documentation. You can also choose to upload the notebook files onto your external page Google Colab and run it. If you have any problems running the provided solutions, please feel free to reach us.