Učenje konvolucijskih modela u miješanoj točnosti

undergraduate thesis

• Title (english): Mixed-precision training of convolutional models
• Author: Dario Oreč
• Mentor: Siniša Šegvić (mentor)
• Committee member: Siniša Šegvić (predsjednik povjerenstva)
• Committee member: Josip Šarić (član povjerenstva)
• Committee member: Ivan Grubišić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2021-07-08, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

U ovom radu su opisane umjetne neuronske mreže, a posebice konvolucijske neuronske mreže. Opisane su metode učenja pomoću gradijentnog spusta i propagacije pogreške unazad, kao i neke tehnike regularizacije. Također je pobliže obrađen rad i korištenje miješane preciznosti, njene prednosti u odnosu na tradicionalni pristup i implementacija u radnom okviru PyTorch. Implementirani su i evaluirani modeli koji koriste miješanu preciznost i oni koji ju ne koriste. Modeli su implementirani pomoću rezidualnih neuronskih mreža, ResNet-18, ResNet-34 i ResNet-50. Na kraju rada su prikazani i uspoređeni rezultati tradicionalnog pristupa i pristupa koji koristi miješanu preciznost.

Abstract (english)

Paper describes artificial neural networks, and in particular convolutional neural networks. Gradient descent learning methods and back-propagation technique are described, as well as some regularization techniques. The work and use of mixed precision, its advantages over traditional approach, and implementation in the PyTorch framework are also discussed in more detail. Models that use mixed precision and those that do not use it have been implemented and evaluated. The models were implemented using residual neural networks, ResNet-18, ResNet-34 i ResNet-50. At the end of the paper, the results of the traditional approach and the approach that uses mixed precision are presented and compared.

• Keywords: neuronske mreže; konvolucijske neuronske mreže; miješana preciznost; amp; ResNet; baseline
• Keywords (english): neural networks; convolutional neural networks; mixed precision; amp; ResNet; baseline
• Language: croatian
• URN:NBN: urn:nbn:hr:168:431599
• Study programme: Title: Computing; Study programme type: university; Study level: undergraduate; Academic / professional title: sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva (sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-14 21:19:48