Nenadzirano učenje optičkog toka

master's thesis

• Title (english): Unsupervised learning of optical flow
• Author: Antonio Pavliš
• Mentor: Siniša Šegvić (mentor)
• Committee member: Siniša Šegvić (predsjednik povjerenstva)
• Committee member: Marko Čupić (član povjerenstva)
• Committee member: Zoran Kalafatić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2021-07-16, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Zadatak procijene optičkog toka jedan je od neriješenih problema u području računalnog vida. Optički tok ima široku primjenu u praksi te se zbog toga pojavljuju brojni pokušaji rješavanja ovog problema. Počevši od formulacije problema kao optimizacijskog postupka te korištenja ručno dizajniranih opisnika, današnje stanje tehnike u području optičkog toka postignuto je dubokim konvolucijskim modelima. Primjer takvog dubokog konvolucijskog modela je RAFT koji se sastoji od tri glavne komponente: kodera značajki, korelacijskog sloja i modula za ažuriranje. Osnovno svojstvo RAFT-a je iterativan pristup i korištenje volumena cijene sa svim parovima piksela. U ovom radu predlažemo kompletan postupak samonadziranog učenja modela RAFT. Učenje obavljamo u dvije faze tako da procijenu toka u nezaklonjenim pikselima iz prve faze koristimo kao točne oznake za vođenje učenja u umjetno zaklonjenim dijelovima tijekom druge faze. Najvažnija karakteristika predloženog postupka je mogućnost učenja na parovima slika bez poznavanja točnog toka čime uklanjamo ograničenja na domenu primjene. Procijenu evaluiramo na skupovima Sintel i Flying Chairs te pokazujemo da naš model postiže rezultate usporedive s najboljim poznatim modelima, a navodimo i mogućnosti budućeg napretka.

Abstract (english)

The task of estimating optical flow is one of the unsolved problems in the field of computer vision. Optical flow has a wide application in practice and therefore there are numerous attempts to solve this problem. Starting from the formulation of the optical flow as an optimization problem and the use of hand-crafted descriptors, the current state-of-the-art in the field of optical flow has been achieved by deep convolutional models. An example of such a deep convolutional model is RAFT which consists of three main components: a feature encoder, a correlation layer and an update operator. The main feature of RAFT is the iterative approach and the use of cost volume with all pairs of pixels. In this paper, we propose a method for self-supervised learning of the RAFT model. We perform the learning in two phases by using the flow estimation in the non-occluded pixels from the first phase as ground-truth to guide learning in the manually occluded areas during the second phase. The proposed method is able to learn on pairs of images without knowing the ground-truth flow, thus removing restrictions on the field of application. We evaluate the estimations at the Sintel and Flying Chairs and show that our model achieves results comparable to the best known models, and we also discuss the possibilities for the future work.

• Keywords: računalni vid; optički tok; duboko učenje; CNN; PyTorch; RAFT; samonadzirano učenje; nenadzirano učenje; analiza kretanja slike; raspoznavanje uzoraka; računalna znanost
• Keywords (english): computer vision; optical flow; deep learning; CNN; PyTorch; RAFT; self-supervised learning; unsupervised learning; image motion analysis; pattern recognition; computer science
• Language: croatian
• URN:NBN: urn:nbn:hr:168:319141
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-13 13:26:48