Sustav za raspoznavanje gesta hrvatskoga znakovnog jezika

master's thesis

• Title (english): System for Recognition of Croatian Sign Language Gestures
• Author: Marko Andreis
• Mentor: Tomislav Hrkać (mentor)
• Committee member: Tomislav Hrkać (predsjednik povjerenstva)
• Committee member: Zoran Kalafatić (član povjerenstva)
• Committee member: Vladimir Čeperić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2023-07-03, Croatia

Abstract

U radu su predložena tri rješenja za zadatak automatskog raspoznavanja gesta hrvatskog znakovnog jezika. Predloženi pristupi temelje se na korištenju konvolucijskih i povratnih neuronskih mreža za analizu slijeda pokreta iz videozapisa. Opisane su osnovne komponente korištenih arhitektura i uspoređeni rezultati na prikupljenom skupu podataka koji uključuje videozapise s pripadajućim oznakama značenja znakova. Prikupljeni videozapisi organizirani su u tri skupa podataka nad kojima su testirane performanse predloženih pristupa. Osnovni model predstavlja lokaliziranje koordinata ključnih točaka na ljudskom tijelu i neuronsku mrežu zasnovanu na ćelijama s dugoročnom memorijom (LSTM). Drugo rješenje arhitektura je neuronske mreže izgrađene od konvolucijskih ćelija s dugoročnom memorijom (ConvLSTM). Posljednji pristup ponuđen je u obliku dugoročne povratne konvolucijske mreže (LRCN).

Abstract (english)

This thesis describes three solutions for the task of automatic recognition of Croatian sign language gestures. The proposed approaches are based on the use of convolutional and recurrent neural networks for motion analysis from videos. The basic components of the used architectures are described and the results are compared on the collected dataset that includes videos with associated sign meaning labels. Collected videos are organized into three datasets on which the performance of the proposed approaches was tested. The baseline model represents the localization of the coordinates of the key points on the human body and neural network based on the long short-term memory cells (LSTM). Second architectural solution is neural network built from convolutional long short-term memory cells (ConvLSTM). The last approach is offered in the form of a long-term recurrent convolutional network (LRCN).

• Keywords: strojno učenje; računalni vid; konvolucijske neuronske mreže; povratne neuronske mreže; znakovni jezik
• Keywords (english): machine learning; computer vision; convolutional neural networks; recurrent neural networks; sign language
• Language: croatian
• URN:NBN: urn:nbn:hr:168:846656
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-10-11 08:16:03