Название: Advanced Techniques in Optimization for Machine Learning and Imaging
Автор: Alessandro Benfenati, Federica Porta, Tatiana Alessandra Bubba, Marco Viola
Издательство: Springer
Год: 2024
Страниц: 173
Язык: английский
Формат: pdf (true), epub
Размер: 55.2 MB
In recent years, non-linear optimization has had a crucial role in the development of modern techniques at the interface of Machine Learning and imaging. The present book is a collection of recent contributions in the field of optimization, either revisiting consolidated ideas to provide formal theoretical guarantees or providing comparative numerical studies for challenging inverse problems in imaging. The covered topics include non-smooth optimisation techniques for model-driven variational regularization, fixed-point continuation algorithms and their theoretical analysis for selection strategies of the regularization parameter for linear inverse problems in imaging, different perspectives on Support Vector Machines trained via Majorization-Minimization methods, generalization of Bayesian statistical frameworks to imaging problems, and creation of benchmark datasets for testing new methods and algorithms. In the past years, Support Vector Machines (SVMs) played a crucial role in the context of Machine Learning, for supervised classification and regression tasks. Even in the Deep Learning era, they can outperform other supervised methods and they are still a popular approach. The paper by A. Benfenati et al. investigates a novel approach by training SVMs via a squared hinge loss functional coupled with sparse-promoting regularization, adopting a Majorization-Minimization method. We perform numerical simulations to compare the performance of most commonly used Langevin Monte Carlo algorithms. The Langevin Monte Carlo (LMC) algorithm (possibly with Metropolis–Hastings adjustment), which is derived from the overdamped Langevin diffusion, has become a popular MCMC method for high-dimensional continuously differentiable distributions since it only requires access to a gradient oracle of the potential of the distribution, which can be computed easily using automatic differentiation softwares such as PyTorch, TensorFlow and JAX.
