Efficient facial expression recognition using decision trees and neural networks

doctoral thesis

• Title (croatian): Učinkovito raspoznavanje izraza lica primjenom stabala odlučivanja i neuronskih mreža
• Author: Ivan Gogić
• Mentor: Igor Sunday Pandžić (mentor)
• Mentor: Jörgen Ahlberg https://orcid.org/0000-0002-6763-5487 (komentor)
• Committee member: Igor Sunday Pandžić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; (Department of Telecommunications); Zagreb
• Defense date and country: 2021, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing
• Universal decimal classification (UDC): 004 - Computer science and technology. Computing. Data processing

Abstract

This thesis investigates a facial expression recognition system that estimates the emotional state of subjects from facial images. Such systems demand accurate and fast algorithms that can run in real-time on platforms with limited computational resources. The proposed algorithms bridge the gap between precise but slow methods and fast but less precise methods, combining decision trees and neural networks. The gentle boost decision trees are trained to extract highly discriminative feature vectors for each facial expression around distinct facial landmark points. These sparse binary features are concatenated to jointly optimize facial expression predictions with a shallow neural network architecture. The joint optimization improves the recognition rates of difficult expressions such as fear and sadness. Since the algorithm depends on accurate landmark locations, a novel face alignment method is introduced using gradient boost decision trees and neural networks organized in a cascaded regression framework. The cascade is initialized by a lightweight convolutional neural network to increase robustness while preserving high efficiency. The thesis begins with an introduction to the problem and the motivation for solving it, followed by an explanation of the theoretical background and a systematic overview of related, previous work. Next, novel algorithms for face alignment and facial expression recognition are described and evaluated on relevant public data sets. The results demonstrate high efficiency and competitive accuracy compared to the state-of-the-art methods suitable for power-efficient applications. The final chapter provides concluding remarks of the thesis.

Abstract (croatian)

Ovaj rad istražuje sustav raspoznavanja izraza lica koji predviđa emotivno stanje subjekata iz slika lica. Takvi sustavi zahtijevaju točne i brze algoritme koji se mogu izvršavati u stvarnom vremenu na platformama sa ograničenim računalnim sredstvima. Predloženi algoritmi premošćuju raskorak između preciznih ali sporih metoda i brzih ali manje preciznih metoda kombinacijom stabala odlučivanja i neuronskih mreža. Stabla odlučivanja učena su pomoću gentle boost metode kako bi izlučila visoko diskriminativni vektor značajki za svaki izraz lica u okolini ključnih točaka lica. Dobiveni prorijeđeni binarni vektori spajaju se kako bi se zajednički optimizirala predviđanja izraza lica pomoću plitke arhitekture neuronske mreže. Zajednička optimizacija poboljšava točnost raspoznavanja za teške izraze kao što su strah i ljutnja. Budući da algoritam ovisi o točnim položajima ključnih točaka lica, predložena je nova metoda detekcije ključnih točaka koja koristi gradient boost stabla odlučivanja i neuronske mreže u okviru kaskadne regresije. Kaskada se inicijalizira pomoću optimizirane konvolucijske neuronske mreže kako bi se poboljšala robusnost metode, a zadržala visoka razina učinkovitosti. Doktorski rad počinje uvodom u problem i motivaciju za njegovo rješavanje, a slijedi ga obrazloženje teoretske pozadine te sistematski pregled povezanog prethodnog istraživanja. Nakon toga, opisani su novi algoritmi za detekciju ključnih točaka i raspoznavanje izraza lica te testirani na relevantnim javnim skupovima podataka. Dobiveni rezultati pokazuju visoku učinkovitost i konkurentnu točnost u usporedbi sa najboljim metodama te prikladnost metode za energetski učinkovite primjene. Posljednje poglavlje zaključuje doktorski rad.

• Keywords: decision trees; neural networks; convolutional neural networks; facial expression recognition; face alignment
• Keywords (croatian): stabla odlučivanja; neuronske mreže; usporedbe intenziteta piksela; poravnavanje lica; detekcija ključnih točaka; raspoznavanje izraza lica
• Language: english
• URN:NBN: urn:nbn:hr:168:882946
• Study programme: Title: Computer Science; Study programme type: university; Study level: postgraduate; Academic / professional title: Doktor znanosti (Doktor znanosti)
• Catalog URL: http://lib.fer.hr/cgi-bin/koha/opac-detail.pl?biblionumber=52611
• Type of resource: Text
• Extent: xiii, 106 str.
• File origin: Born digital
• Access conditions: Open access
• Created on: 2022-04-27 11:43:39