Métodos estocásticos de optimización

The main purpose of this work is to study stochastic optimization methods used to minimize the problem of finite sums for convex functions. Within these methods, the method of gradient descent with mini-batch and the stochastic version of the Frank-Wolfe method (or Conditional Gradient) are specifically exposed in the work. For the gradient descent method with mini-batch, a limit was shown on the number of iterations necessary to converge, which depends on the size of the batch used in the algorithm. Later an optimal size of this was found where the number of iterations necessary for convergence is minimized. On the other hand, in the part dedicated to the study of the new stochastic version of the Frank-Wolfe algorithm, the operation of the algorithm was explicitly shown. In addition, lemmas and theorems were demonstrated that guarantee their convergence in a time less than or equal to the original method. The algorithm's stop method was redefined, showing that it is a stochastic version equivalent to the original stop method. Finally, the algorithms were implemented with real data using the loss functions of the logistic and linear regressions as the functions of the problem stated at the beginning of this summary. To evaluate the performance of these algorithms, they were compared with stochastic noise reduction algorithms known as SAGA and SVRG.

• Autores: Guerrero Santaren, Daniel Mauricio
• Tipo de recurso: Trabajo de grado de pregrado
• Fecha de publicación: 2020
• Institución: Universidad de los Andes
• Repositorio: Séneca: repositorio Uniandes
• Idioma: spa
• OAI Identifier: oai:repositorio.uniandes.edu.co:1992/51296
• Acceso en línea: http://hdl.handle.net/1992/51296
• Palabra clave: Optimización matemática; Funciones convexas; Métodos iterativos (Matemáticas); Programación estocástica; Aproximación estocástica; Matemáticas
• Rights: openAccess
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/