Many tasks in natural language processing, computer vision, and computational biology involve predicting structured outputs. Researchers are increasingly applying deep representation learning to these problems, but the structured component of these approaches is usually quite simplistic. For example, neural machine translation systems use unstructured training of local factors followed by …
Contrary to classical bias/variance trade-offs, deep learning practitioners have observed that vastly overparameterized neural networks with the capacity to fit virtually any labels nevertheless generalize well when trained on real data. One possible explanation of this phenomenon is that complexity control is being achieved by implicitly or explicitly controlling the magnitude of …
The multi-armed bandit problem is a common sequential decision-making framework where at each time step a player selects an action and receives some reward from selecting that action. The aim is to select actions to maximize the total reward. Commonly it is assumed that the (expected) reward of each action …
Modeling and tractable computation form two fundamental but competing pillars of data science; indeed, fitting models to data is often computationally challenging in modern applications. At the same time, a “good” model is one that imposes the right kind of structure on the underlying data-generating process, and this involves trading …
Machine learning algorithms for policy and decision making are becoming ubiquitous. In many societal applications, the inferences we can draw are often severely limited not by the number of subjects in the data but rather by limited observations available for each subject. My research focuses on tackling these limitations both …
The earth mover’s distance (EMD) is a scalar measure of dissimilarity between histograms. Introduced over 200 years ago, the EMD has played a central role in linear programming, information retrieval, and is emerging as useful objective in machine learning. During the 1990’s, the EMD was generalized from a functional that acts …
Uncertainty penetrates in every corner of machine learning, ranging from data and adversary uncertainty, model uncertainty, all the way to dynamics uncertainty, even task uncertainty, and beyond. When faced with complicated machine learning tasks under various forms of uncertainty, the traditional empirical risk minimization framework, along with the rich off-the-shelf …
Through recent advances in machine learning (ML) technology, we are getting closer to realizing the broadly stated goal of “arti cially intelligent”, autonomous agents. In many cases — like cognitive radio, swarm robotics, and e-commerce — these agents will not be acting in isolation, and it is critical for them to …
Billions of passwords are compromised in the last few years and are available online. Attackers are using these leaked passwords to compromise user accounts on other not-yet-leaked web services. Because many users reuse their passwords across different web services, such attacks are the most imminent threat to users’ account security …
Abstract: This talk will discuss a variety of recent papers, mostly from 2019. The talk will be organized around four “themes” of deep learning: Domain Independence Information theory Moore’s Law of AI Self-Supervised Pretraining Vector Quantization Mutual Information Objectives