Large photo mj small

Optimization, Learning and Systems

Martin Jaggi

Recorded 15 February 2016 in Lausanne, Vaud, Switzerland

Event: IC Colloquia - EPFL IC School Colloquia

Abstract

The massive growth of available data has moved data analysis and machine learning to center stage in many industrial as well as scientific fields, ranging from astronomy to health science, web and sensor data, and countless other applications. In this talk, we focus on the computational challenges of machine learning on large datasets through the lens of mathematical optimization.

Significant recent research aims to improve the efficiency, scalability, and theoretical understanding of iterative optimization algorithms used for training machine learning models. Motivated by widely used regression and classification techniques, we discuss new results for first-order optimization techniques, empowering them with primal-dual certificates and convergence guarantees which are valuable for practitioners. We also illustrate how Frank-Wolfe and coordinate descent algorithms can help to trade-off accuracy against complexity (measured for example in sparsity or rank) of machine learning models.

While single machine solvers have been highly optimized, they can not easily be transferred to the distributed setting, i.e. when the dataset exceeds the storage capacity of a single computer. For the users of machine learning methods, this lack of generality of learning algorithms is becoming increasingly frustrating, as the complexity and variability of the underlying systems is increasing, for example in terms of differences in communication, computation and memory speeds.
This highlights the necessity for new learning algorithms which are able to efficiently adapt to the available compute environment, spanning a wide range from cheap public cloud computing stacks to more traditional HPC systems. At the same time, the theoretical efficiency guarantees should ideally be adaptive to the real system properties as well. Finally, open source optimization software combined with public benchmarks can help industrial users navigate the increasingly complex landscape of commercial big data software frameworks.

Watched 9397 times.

 Watch