Varun Jog – SILO

Geometric tools in information theory

Video: https://vimeo.com/184352560 Concepts in geometry often have parallels in information theory; for example, volume and entropy, surface area and Fisher information, sphere-packing and channel coding, and Euclidean balls and Gaussian distributions, to name a few. These similarities provide a simple way to posit theorems in one area by translating the …

Information theoretic perspectives on learning algorithms

In statistical learning theory, generalization error is used to quantify the degree to which a supervised machine learning algorithm may overfit to training data. We overview some recent work [Xu and Raginsky (2017)] that bounds generalization error of empirical risk minimization based on the mutual information I(S;W) between the algorithm …