Prepoznavanje abecede znakovnog jezika koristeći metode strojnog učenja

undergraduate thesis

• Title (english): Machine Learning Methods for Sign Language Alphabet Recognition
• Author: Lucija Mičić
• Mentor: Dario Bojanjac (mentor)
• Committee member: Dario Bojanjac (predsjednik povjerenstva)
• Committee member: Jelena Božek (član povjerenstva)
• Committee member: Leo Vincelj (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2022-07-08, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Jednoručna abeceda hrvatskog znakovnog jezika sastoji se od slova sa statičnim i dinamičnim pokretima. Prvi korak u izradi bio je osmišljavanje početnog skupa podataka, odnosno koordinata. Zatim je klasifikacija navedenih znakova implementirana metodama strojnog učenja, točnije, modelom neuronske mreže i metodama radnog okvira MediaPipe Hands za izdvajanje ruke na ulaznim slikama kamere i ocrtavanje glavnih točaka dlana. Poseban naglasak stavljen je na analizu koordinata za prepoznavanje znakova s dinamičnim pokretom, koji, u početnom trenutku, najčešće nalikuju nekom od slova sa statičnim pokretom iz kojih su se prepoznavali. Svrha razvoja aplikacije bila je premostiti jezičnu barijeru s kojom se osobe s oštećenjima sluha svakodnevno susreću i time unaprijediti opcije pristupačnosti.

Abstract (english)

The single-handed alphabet of Croatian sign language consists of letters with static and dynamic movements. The first step in the development was to design the initial set of data, ie coordinates. Then the classification of these signs was implemented by machine learning methods, more precisely, the neural network model and the frameworks methods MediaPipe Hands to separate the hand on the camera input images and delineate the main points of the palm. Special emphasis was placed on the analysis of coordinates for recognizing signs with dynamic movement, which, at the initial moment, most often resemble one of the letters with static movement from which they were recognized. The purpose of the application development was to bridge the language barrier that people with hearing impairments face on a daily basis and thus improve accessibility options.

• Keywords: hrvatski znakovni jezik; abeceda; strojno učenje; neuronske mreže; MediaPipe Hands
• Keywords (english): Croatian sign language; alphabet; machine learning; neural networks; MediaPipe Hands
• Language: croatian
• URN:NBN: urn:nbn:hr:168:521886
• Study programme: Title: Computing; Study programme type: university; Study level: undergraduate; Academic / professional title: sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva (sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-15 10:50:45