Kontrastno učenje modela za prevođenje slika

undergraduate thesis

• Title (english): Contrastive learning for image-to-image translation
• Author: Bernard Spiegl
• Mentor: Siniša Šegvić (mentor)
• Committee member: Siniša Šegvić (predsjednik povjerenstva)
• Committee member: Josip Šarić (član povjerenstva)
• Committee member: Ivan Grubišić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2021-07-08, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Modeli za prevođenje slika pretvaraju sliku iz izvorne domene u odredišnu domenu i pri tome nastoje sačuvati sadržaj slike.
Metoda s kontrastivnim učenjem jedan je od pristupa za rješavanje ovakvog tipa problema.
Prednost ove metode je što omogućuje treniranje u slučaju kada je domena samo jedna slika.
Ono što ju razlikuje od prethodnika je činjenica da koristi isječke slika prilikom treniranja umjesto cijelih slika.
Također je pokazano da uzimanje negativa (isječci slike potrebni za izračunavanje gubitka) iz iste slike postiže bolje rezultate nego kad se negativi uzimaju iz drugih slika u datasetu.
Ovakav pristup potiče mapiranje odgovarajućih isječaka na približno isto mjesto u odnosu na druge isječke (negative) dok istovremeno poboljšava kvalitetu rezultirajuće slike i smanjuje vrijeme potrebno za treniranje u odnosu na baseline metode: CycleGAN, MUNIT, DRIT, DistanceGAN, SelfDistance i GCGAN.

Abstract (english)

Image-to-image translation models transfer images from input domain to output domain in an endeavour to retain the content of the image.
Contrastive Unpaired Translation is one of the existing methods for solving such problems.
Significant advantage of this method, compared to competitors, is the ability to train and perform well in cases where both input and output domains are only a single image.
Another key thing that differentiates this method from its predecessors is the usage of image patches rather than whole images.
It also turns out that sampling negatives (patches required to calculate loss) from the same image achieves better results than a scenario where the negatives are sampled from other images in the dataset.
This type of approach encourages mapping of corresponding patches to the same location in relation to other patches (negatives) while at the same time both improves the output image quality and significantly decreases the time required to train the model compared to other baseline methods such as CycleGAN, MUNIT, DRIT, DistanceGAN, SelfDistance and GCGAN.

• Keywords: duboko učenje; prevođenje slika; kontrastno učenje; konvolucijske neuronske mreže; generativne suparničke mreže; generiranje slika
• Keywords (english): deep learning; image-to-image translation; contrastive learning; convolutional neural networks; generative adversarial networks; image generation
• Language: english
• URN:NBN: urn:nbn:hr:168:878648
• Study programme: Title: Computing; Study programme type: university; Study level: undergraduate; Academic / professional title: sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva (sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-14 21:40:23