| CODE | SOR3351 | ||||||||||||||||
| TITLE | Dynamic Programming and Optimal Control | ||||||||||||||||
| UM LEVEL | 03 - Years 2, 3, 4 in Modular Undergraduate Course | ||||||||||||||||
| MQF LEVEL | 6 | ||||||||||||||||
| ECTS CREDITS | 4 | ||||||||||||||||
| DEPARTMENT | Statistics and Operations Research | ||||||||||||||||
| DESCRIPTION | In this study-unit, we are going to discuss topics related to Dynamic Programming (DP) and Optimal Control (OC). DP is a collection of methods for solving sequential decision problems by decomposing a multistage problem into a sequence of interrelated one-stage problems. Fundamental to this decomposition is the principle of optimality, which was developed by Richard Bellman in the 1950s, while its importance is that an optimal solution for a multistage problem can be found by solving a functional equation relating the optimal value for a (t + 1)-stage problem to the optimal value for a t-stage problem. DP is considered to be more of an approach, a way of thinking about a problem, rather than a fixed mathematical statement in which a problem is cast. Moreover, in this study-unit, we are also going to discuss the more general problem posed in OC which is to find a "control" for a dynamical system over a period of time such that an objective function is optimized. Many applications and examples, drawn from a broad variety of fields, will be discussed. This study-unit will be particularly useful for students that want to pursue a career in Operations Research and/or Economics. Contents: - The Dynamic Programming Algorithm; - Deterministic Systems, the Shortest Path Problem and Applications; - Deterministic Continuous-Time Optimal Control; - The Hamilton-Jacobi-Bellman Equation; - Pontryagin's Maximum Principle; - Linear Systems and Quadratic Cost; - Problems with Perfect and Imperfect State 福利在线免费; - Certainty Equivalent Controllers; - Adaptive Control; - Introduction to Infinite Horizon Problems. Study-Unit Aims: This study-unit aims at giving a rigorous treatment to the mathematical formulation and solution methods used in deterministic Optimal Control (OC). There are various applications of OC in Operations Research including inventory control, dynamic portfolio analysis, optimal stopping problems, scheduling problems. This study-unit aims at providing students an appreciation of such real-world OC problems and how these can be formulated as mathematical models. In doing so students will be equipped with the necessary knowledge needed to use this topic in the relevant area of interest. Learning Outcomes: 1. Knowledge & Understanding: By the end of the study-unit the student will be able to: - Demonstrate through mathematical programming the use of Dynamic Programming (DP) in several real-life settings; - Identify all components of a DP model; - Describe and Formulate real-life problems using the DP framework; - Explain the principles of finite-dimensional dynamic optimization; - Formulate various algorithms for solving a dynamic program; - Analyse the optimality of a given control law. 2. Skills: By the end of the study-unit the student will be able to: - Define and Model sequential decision making problems; - Identify when to model a problem as a DP problem; - Apply DP algorithms to solve dynamic optimization problems; - Apply calculus of variations to find extrema of functionals; - Design optimal controllers; - Interpret the results in the given decision making context; - Use software packages to apply DP and OC methods. Main Text/s and any supplementary readings: Main Texts: - Bertsekas, Dimitri P. Dynamic Programming and Optimal Control. Athena Scientific, 2001. Print. - Bertsekas, Dimitri P. Dynamic Programming: Deterministic and Stochastic Models. Prentice-Hall, 1987. Print. - Sethi, Suresh P. Optimal Control Theory Applications to Management Science and Economics. 3rd Ed. 2019.. ed. 2019. Web. - Sniedovich, Moshe. Dynamic Programming: Foundations and Principles. CRC, 2010. Monographs and Textbooks in Pure and Applied Mathematics. Web. - Bensoussan, A. Dynamic Programming and Inventory Control. IOS, Incorporated, 2011. Web. Supplementary Readings: - Bertsekas, Dimitri P. Reinforcement Learning and Optimal Control. Athena Scientific, 2019. Print. - Anderson, Brian D.O., and John B. Moore. Optimal Control: Linear Quadratic Methods. Dover, 2007. Print. Dover Books on Engineering. - Longo, Stefano. Optimal and Robust Scheduling for Networked Control Systems. 1st ed. Vol. 4. Boca Raton, Fla: Taylor & Francis, 2013. Automation and Control Engineering. Web. - Reble, Marcus. Model Predictive Control for Nonlinear Continuous-Time Systems with and without Time-Delays. Berlin: Logos Verlag, 2013. Web. |
||||||||||||||||
| ADDITIONAL NOTES | Pre-requisite Study-units: SOR1310, SOR1320, SOR2330 | ||||||||||||||||
| STUDY-UNIT TYPE | Lecture and Tutorial | ||||||||||||||||
| METHOD OF ASSESSMENT |
|
||||||||||||||||
| LECTURER/S | Maria Kontorinaki |
||||||||||||||||
|
The University makes every effort to ensure that the published Courses Plans, Programmes of Study and Study-Unit information are complete and up-to-date at the time of publication. The University reserves the right to make changes in case errors are detected after publication.
The availability of optional units may be subject to timetabling constraints. Units not attracting a sufficient number of registrations may be withdrawn without notice. It should be noted that all the information in the description above applies to study-units available during the academic year 2025/6. It may be subject to change in subsequent years. |
|||||||||||||||||