Métodos de otimização aplicados à estatística

Abstract

In this thesis, we study an intersection between the areas of optimization methods and the statistical models. We work in this intersection in two ways. First, we take as a case study, in the field of optimization methods, the gradient descent, accelerated gradient descent, high-order accelerated gradient and Newton-Raphson and we study the theoretical convergence rates of the sequences generated by these methods. Second, we take as a case study, in the field of statistical models, the generalized linear models and studies their estimation process via logarithm of the likelihood function. Finally, to consolidate the intersection between these areas, we implemented the maximum likelihood estimators for the logistic regression model. Through the empirical statistical analysis of the optimization methods under study, we concluded that the high-order accelerated gradient method, when it uses information from one derivative of the objective function, has an empirical performance superior to the other methods of first order when we look to the empirical rates of convergence and the run times, despite theoretically having a convergence rate lower than the Newton-Raphson method. Thus opening the possibility of future work to an analytical investigation of this phenomenon.