Gusta predikcija primjenom slojeva pažnje

master's thesis

• Title (english): Dense prediction by means of self-attention layers
• Author: Leo Pleše
• Mentor: Siniša Šegvić (mentor)
• Committee member: Siniša Šegvić (predsjednik povjerenstva)
• Committee member: Marko Čupić (član povjerenstva)
• Committee member: Jan Šnajder (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2022-07-14, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Gusta predikcija složen je izazov u računalnom vidu koji, uz konvolucijske modele, sve više rješavaju i transformeri. Motivirani time analiziramo generalizacijsku moć u gustoj predikciji i vremensko-prostornu učinkovitost transformera i konvolucijskih modela. Od transformera za gustu predikciju odabiremo Segmenter i SWIN, a za konvolucijski model primarno piramidalni SwiftNet. Inicijalno učenje Segmentera na PASCAL Contextu indiciralo je generalizacijsku moć Segmentera. Pokazali smo na Cityscapesu da su transformeri i konvolucijski modeli usporedive točnosti, no da su transformeri zahtjevniji za učenje. Na ADE20k smo otkrili da je djelomično naučen SWIN-T bolje točnosti od Segmenter-Ti i na Cityscapesu da nevalidiran SWIN-B bolje generalizira od Segmenter-B. Isprobali smo na Cityscapesu nekoliko ansambala i združenih modela SwiftNet-ResNet18-p i Segmenter-Ti kako bismo dobili model bolje točnosti od zasebnih modela. Ansambli su u tome uspjeli, a združeni modeli nisu. Vjerujemo da bi se iscrpnom validacijom našli združeni modeli koji bi premašili točnost ansambala.

Abstract (english)

Dense prediction is a complex challenge in computer vision that, besides convolutional models, is increasingly being solved by transformers. Motivated by this, we analyze the generalization power in dense prediction and the spatio-temporal efficiency of transformers and convolutional models. From transformers for dense prediction we choose Segmenter and SWIN, and for convolutional model primarily pyramidal SwiftNet. The initial learning of Segmenters on PASCAL Context indicated generalization power of Segmenters. We showed on Cityscapes that transformers and convolutional models are of comparable accuracy but transformers are more demanding to learn. On ADE20K we found that the partially learned SWIN-T had better accuracy than Segmenter-Ti and on Cityscapes that the unvalidated SWIN-B generalizes better than Segmenter-B. We tested couple ensembles and joint models of SwiftNet-ResNet18-p and Segmenter-Ti on Cityscapes so as to get a model of better accuracy than individual models. The ensembles succeeded in this but the joint models did not. We believe that by exhaustive validation joint models which would exceed ensembles' accuracy could be found.

• Keywords: duboko učenje; računalni vid; gusta predikcija; transformeri; pažnja; konvolucijski modeli; segmentacijska točnost; vremenska učinkovitost; prostorna učinkovitost; Segmenter; SWIN transformer; piramidalni SwiftNet; PASCAL Context; ADE20K; Cityscapes
• Keywords (english): deep learning; computer vision; dense prediction; transformers; attention; convolutional models; segmentation accuracy; time efficiency; spatial efficiency; Segmenter; SWIN transformer; pyramidal SwiftNet; PASCAL Context; ADE20K; Cityscapes
• Language: croatian
• URN:NBN: urn:nbn:hr:168:060616
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-13 15:54:16