Metode gradijentnog spusta u strojnom učenju

undergraduate thesis

• Title (english): Gradient descent methods for machine learning
• Author: Nora Ivić
• Mentor: Josipa Pina Milišić (mentor)
• Committee member: Josipa Pina Milišić (predsjednik povjerenstva)
• Committee member: Tomislav Burić (član povjerenstva)
• Committee member: Mario Krnić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2024-09-11, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

U ovom završnom radu dan je pregled optimizacijskih metoda koje se temelje na algoritmu gradijentnog spusta te je diskutirana njihova brzina konvergencije. Za bezuvjetnu minimizaciju glatkih konveksnih funkcija obrađena je metoda gradijentnog spusta, kao metoda prvog reda, i Newtonova metoda, kao metoda drugog reda. Nadalje, kako bismo proširili područje primjene gradijentnog spusta, za generalizaciju na nediferencijabilne Lipshitz-kontinuirane funkcije prikazana je metoda subgradijentnog... More spusta. Kako su ove metode u praksi često neupotrebljive zbog velike vremenske složenosti, napravljen je prikaz nekoliko akceleriranih metoda koje se temelje na gradijentnom spustu, od kojih je detaljnije opisana metoda stohastičkog gradijentnog spusta. Less

Abstract (english)

This final thesis gives an overview of optimization methods based on the gradient descent algorithm and their convergence rates. For the unconstrained optimization of smooth convex functions, the gradient descent method is presented as a first-order method and the Newton method as a second-order method. In order to extend the range of application of gradient descent, the subgradient descent method is also given for generalization to non-differentiable Lipshitz continuous functions. Since... More these methods are often unusable in practice due to the high time complexity, several accelerated methods based on gradient descent are presented, of which the stochastic gradient descent method is described in more detail.

Less

• Keywords: iterativne metode; bezuvjetna minimizacija; gradijentni spust; subgradijentni spust; stohastički gradijentni spust; brzina konvergencije
• Keywords (english): iterative methods; unconstrained minimization; gradient descent; subgradient descent; stochastic gradient descent; rate of convergence
• Language: croatian
• URN:NBN: urn:nbn:hr:168:324978
• Study programme: Title: Computing; Study programme type: university; Study level: undergraduate; Academic / professional title: sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva (sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Open access
• Created on: 2024-10-12 20:53:32