Lokalizacija objekata odozdo prema gore primjenom Fisherovih vektora

doctoral thesis

• Title (english): Bottom-up object localization with Fisher vectors
• Author: Valentina Zadrija
• Mentor: Siniša Šegvić (mentor)
• Committee member: Siniša Šegvić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; (Department of Electronics, Microelectronics, Computer and Intelligent Systems); Zagreb
• Defense date and country: 2017, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Electrical Engineering; Electronics
• Universal decimal classification (UDC): 621.3 - Electrical engineering

Abstract

U okviru ove disertacije razmatraju se problemi klasifikacije slika i lokalizacije objekata u složenim scenama. Posebna je pažnja usmjerena učenju lokalizacijskih modela uz slabi nadzor budući da se na taj način zaobilazi vremenski zahtjevan proces označavanja lokacija objekata u slikama za učenje. U fazi učenja, dostupne su isključivo oznake prisutnosti objekta u slici, dok se u fazi testiranja zahtjeva predikcija lokacija objekata u vidu opisanih poligona kao i u slučaju učenja pod strogim nadzorom. Predstavljen je pristup za lokalizaciju objekata temeljen na reprezentaciji Fisherovim vektorima i slabo nadziranom učenju rijetkih lokalizacijskih modela. Predstavljanje slike i slikovnih okana Fisherovim vektorima omogućava primjenu lokalizacijskog modela učenog nad opisnicima cjelokupnih slika za proračun odziva slikovnih okana. Pomoću lokalizacijskih modela rijetkih po komponentama umanjuje se utjecaj prenaučenosti i omogućava učinkovit proračun odziva slikovnog okna. Za poboljšanje lokalizacijske točnosti predložena je primjena metričke normalizacije po komponentama Fisherovog vektora slike. Kako bi se omogućila vremenski efikasna primjena normalizacija u fazi lokalizacije, doprinosa slikovnog okna odzivu normalizirane slike određuje se aproksimacijom prvog reda. Naposljetku, budući da Fisherovi vektori ne uzimaju u obzir prostorne odnose okana u slici, predložene su reprezentacije lokalnog prostornog rasporeda slikovnih riječi u vidu prostornih Fisherovih vektora i prostornih histograma. Razvijeni su algoritmi kojima se na temelju slikovnih okana pozitivnog odziva generiraju predikcije lokacija objekata u vidu opisanih poligona. Provedeno je iscrpno eksperimentalno vrednovanje opisanog pristupa na problemima lokalizacije prometnih znakova i pješačkih prijelaza u složenim prometnim scenama. Pokazano je da se opisani pristup lokalizaciji može primijeniti za potrebe automatizacije digitalnog kartiranja. Za lokalizaciju pješačkih prijelaza, predstavljen je nov skup podataka dobiven polu-automatski na temelju dobrovoljno prikupljenih geopodataka iz OpenStreetMap karte i georeferenciranog videa. Eksperimentalni rezultati pokazuju da predloženi lokalizacijski i reprezentacijski modeli postižu iznimno dobre rezultate unatoč različitim obilježjima traženih objekata i učenju uz slabi nadzor.

Abstract (english)

In this thesis, we consider the problems of image classification and object localization. In particular, we focus on the problem of the weakly supervised object localization in complex scenes. In such setting, the time-consuming step of annotating the object locations in the training set is sidestepped and a localization model is learned by using image-wide labels. At the test time, however, bounding boxes have to be predicted for each learned object class as in the strongly supervised case. We have proposed a novel weakly supervised localization method based on Fisher vector image representation and model sparsity at the component level. The Fisher embedding allows weakly supervised training of localization models by employing image-wide labels. The obtained model is then applied in a sliding window manner to determine the patches responsible for the image label. Model sparsity reduces overfitting and enables fast evaluation of the patch score. In order to generate object predictions, we have proposed two novel methods which allow us to generate bounding polygons from the patches with a positive score. In order to improve the localization performance, we have proposed to use intra-component metric normalization in conjunction with the component-level sparsity. We have also introduced a first-order approximation of the normalized Fisher vector score to make our approach compatible with non-linear normalizations. The proposed approximation allows us to determine the patch-level contribution by a simple dot product. Finally, due to the fact that the Fisher vectors represent an image as an orderless collection of patch descriptors, we have proposed two novel representations of pairwise patch layout: spatial histograms and spatial Fisher vectors. We have performed an extensive evaluation of the proposed method for the tasks of traffic sign and pedestrian crossing localization. The experiments show that our method can be used for the purpose of the road-environment mapping. For the task of pedestrian crossings localization, we have introduced a novel dataset containing 2381 images of pedestrian crossings obtained by semi-automated matching of OpenStreetMap data (longitude, latitude) to GPS references of video frames. The experiments indicate that the proposed method is able to deliver reliable performance, despite weak supervision and different qualities of the involved object classes. Keywords: object localization, Fisher vectors, weakly supervised learning, sparse localization models, spatial layout representations, computer vision, machine learning.

• Keywords: lokalizacija objekata; Fisherovi vektori; slabo nadzirano učenje; rijetki modeli; model prostornog rasporeda; računalni vid; strojno učenje.
• Keywords (english): object localization; Fisher vectors; weakly supervised learning; sparse localization models; spatial layout representations; computer vision; machine learning.
• Language: croatian
• URN:NBN: urn:nbn:hr:168:316032
• Study programme: Title: Electrical Engineering and Computing; Study programme type: university; Study level: postgraduate; Academic / professional title: Doktor znanosti elektrotehnike i računarstva (Doktor znanosti elektrotehnike i računarstva)
• Type of resource: Text
• Extent: 134 str. ; 30 cm.
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2019-04-05 13:50:43