For stochastic control problems with uncountable state and action spaces, the computation of optimal policies is known to be prohibitively hard. In this talk, we will present conditions under which finite models obtained through quantization of the state and action sets can be used to construct approximately optimal policies. Under further conditions, we obtain explicit rates of convergence to the optimal cost of the original problem as the quantization rate increases. We study various conditions on the controlled system models such as weak continuity, strong continuity and continuity in total variation, and the state and action spaces. Using information theoretic arguments, we show that the convergence rates are order-optimal for a large class of problems.
We then extend our analysis to decentralized stochastic control problems, also known as team problems, which are increasingly important in the context of networked control systems. We present some existence results for optimal policies, and building on these we show that for a large class of sequential dynamic team problems one can construct a sequence of finite models obtained through the quantization of measurement and action spaces whose solutions constructively converge to the optimal cost. The celebrated counterexample of Witsenhausen is an important special case that will be discussed in detail. (Joint work with Naci Saldi and Tamas Linder).
Video: https://vimeo.com/163266798