In this talk, we will explore the implicit bias of generic optimization methods and its connection to generalization in ill-posed optimization problems. We will specifically study optimizing underdetermined linear regression or separable linear classification problems using common optimization methods including, mirror descent, natural gradient descent and steepest descent with respect …
In subset selection we search for the best linear predictor that involves a small subset of variables. Due to the vast applicability of this model, many approaches have been proposed by different communities, including enumeration, greedy algorithms, branch and bound, and convex relaxations. Our point of departure is to understand …
Machine teaching provides a rigorous formalism for a number of real-world applications including personalized educational systems, adversarial attacks, and program synthesis by examples. In this talk, I will discuss research questions in the context of two societal applications: (i) teaching participants of citizen science projects for biodiversity monitoring, and (ii) …
The advent of data-hungry applications has enabled computers to interpret what they see, communicate in natural language, and answer complex questions. There is a hidden catch, however: the reliance of all these state-of-the-art systems on high-effort tasks like data preparation and data cleaning. It is estimated that 70% to 80% …
We present a mathematical analysis of a non-convex energy landscape for Robust Subspace Recovery. Under a deterministic condition, the only stationary point in a large neighborhood of an underlying subspace is the subspace itself. The same deterministic condition implies that a geodesic gradient descent method can exactly recover the underlying …
Integration against an intractable probability measure is among the fundamental challenges of Bayesian inference. A useful approach to this problem seeks a deterministic coupling of the measure of interest with a tractable “reference” measure (e.g., a standard Gaussian). This coupling is induced by a transport map, and enables direct simulation …
In statistical learning theory, generalization error is used to quantify the degree to which a supervised machine learning algorithm may overfit to training data. We overview some recent work [Xu and Raginsky (2017)] that bounds generalization error of empirical risk minimization based on the mutual information I(S;W) between the algorithm …
Willem’s Abstract: Atmospheric light detection and ranging (lidar) provides a unique capability to resolve atmospheric vertical structures of clouds and aerosols with very high sensitivity at high altitude and temporal resolutions. A variety of lidar instruments exist which focus on the measurement of the optical properties of the atmosphere in …
With the availability of huge amounts of unlabeled data, unsupervised learning methods are gaining increasing popularity and importance. We focus on ”unsupervised ensemble learning”, where one obtains the predictions of multiple classifiers over a set of unlabeled instances. The classifiers may be human experts as in crowdsourcing, or prediction algorithms …
High-dimensional high-order arrays, or tensors, arise in many modern scientific applications including genomics, brain imaging, and social science. In this talk, we consider two specific problems in tensor data analysis: low-rank tensor completion and tensor SVD. We first propose a framework for low-rank tensor completion via a novel tensor measurement …