SILO: Variational inference – reconciling statistical and convergence guarantees

As a computational alternative to Markov chain Monte Carlo approaches, variational inference (VI) is becoming increasingly popular for approximating intractable posterior distributions in large-scale Bayesian models due to its comparable efficacy and superior efficiency. Several recent works provide theoretical justifications of VI by proving its statistical optimality for parameter estimation under various settings; meanwhile, formal analysis on the algorithmic convergence aspects of VI is still largely lacking.  I shall talk about a general theory to show how the choice of variational family is critical to good statistical performance of the algorithmic solution. I shall also present a few case studies, caution against potential pitfalls, and offer remedies.