Scene illumination color estimation methods based on convolutional neural networks

doctoral thesis

• Title (croatian): Metode procjene boje osvjetljenja scene zasnovane na konvolucijskim neuronskim mrežama
• Author: Karlo Koščević
• Mentor: Sven Lončarić (mentor)
• Committee member: Sven Lončarić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; (Department of Electronic Systems and Information Processing); Zagreb
• Defense date and country: 2022, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing
• Universal decimal classification (UDC): 004 - Computer science and technology. Computing. Data processing

Abstract

Different objects in the same scene emit the incident light differently and are perceived differently colored. The human visual system is very robust to changes in the scene and adapts rapidly, and can perceive objects' colors invariant of the light source. Image sensors in digital cameras do not have this ability, and thus a pre-processing step is dedicated to achieving invariance of colors to the scene illumination. That step is referred to as computational color constancy and consists of illumination estimation and chromatic adaptation. The focus of this thesis is on illumination estimation, and the aim is the analysis and implementation of various approaches to illumination estimation using deep learning methodology. Deep neural networks have exceptional generalization capability and achieve state-of-the-art results in computer vision. However, deep learning usually involves the question of large datasets that are not typical in illumination estimation. Therefore, a part of the research in the thesis is dedicated to defining the methodology for creating a dataset for illumination estimation, and a novel dataset is presented.The scientific contribution of the doctoral thesis is divided into four parts: illumination color estimation method based on a convolutional neural network with an attention mechanism; illumination color estimation method using scene lighting classification based on deep learning; illumination color estimation method based on recurrent deep neural network with a multistage loss function; the Cube++ illumination estimation dataset.

Abstract (croatian)

Različiti objekti u istoj sceni drugačije emitiraju upadnu svjetlost te se percipiraju kao različito obojeni. Ljudski vizualni sustav je vrlo robustan na promjene u sceni i brzo se prilagođava te može razaznati boje objekata neovisno o osvjetljenju. Senzori za stvaranje slike u digitalnim fotoaparatima nemaju tu mogućnost te stoga na početku procesa formiranja digitalne slike postoji korak predobrade za postizanju invarijantnosti boja na osvjetljenje u sceni. Taj korak naziva se računalna postojanost boja i sastoji se od procjene osvjetljenja i kromatske adaptacije. Fokus ove disertacije je na procjeni osvjetljenja, a cilj istraživanja je analiza i implementacija različitih pristupa za procjenu osvjetljenja korištenjem metodologije dubokog učenja. Duboke neuronske mreže imaju iznimnu sposobnost generalizacije i postižu vrhunske rezultate u računalnom vidu. Međutim, duboko učenje obično povlači i pitanje velikih skupova podataka, netipičnih za procjenu osvjetljenja. Stoga je dio istraživanja ove disertacije posvećen definiranju metodologije za izradu skupa podataka za procjenu osvjetljenja te je prezentiran novi skup podataka.Znanstveni doprinos disertacije podijeljen je u četiri dijela: metoda procjene boje osvjetljenja zasnovana na konvolucijskoj neuronskoj mreži s mehanizmom pažnje; metoda procjene boje osvjetljenja korištenjem klasifikacije osvjetljenja scene zasnovana na dubokom učenju; metoda procjene boje osvjetljenja zasnovana na povratnoj dubokoj neuronskoj mreži s višestupanjskom funkcijom gubitka; skup podataka za procjenu osvjetljenja Cube++.

• Keywords: color constancy; convolutional neural networks; deep learning; illumination estimation; white balancing
• Keywords (croatian): postojanost boja; konvolucijske neuronske mreže; duboko ucenje; procjena osvjetljenja; podešavanje bijele
• Language: english
• URN:NBN: urn:nbn:hr:168:694905
• Promotion: 2022
• Study programme: Title: Computer Science; Study programme type: university; Study level: postgraduate; Academic / professional title: Doktor znanosti (Doktor znanosti)
• Type of resource: Text
• Extent: ix, 124 str. : ilustr. u bojama
• File origin: Born digital
• Access conditions: Open access
• Created on: 2022-05-23 10:10:21