Prepoznavanje gesti ruku

undergraduate thesis

• Title (english): Hand gesture recognition
• Author: Bjanka Vrljić
• Mentor: Ivica Botički (mentor)
• Committee member: Ivica Botički (predsjednik povjerenstva)
• Committee member: Mirjana Domazet-Lošo (član povjerenstva)
• Committee member: Juraj Dončević (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2022-07-08, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Završni rad obuhvaća izradu sustava za prepoznavanje slova abecede američkog znakovnog jezika u stvarnom vremenu, što uključuje detekciju ruke na kameri te izradu i korištenje klasifikatora gesti. Detekcija ruke je implementirana pomoću biblioteke MediaPipe. MediaPipe crta pojednostavljeni kostur ruke na ekranu i detektira ključne točke ruke. Te ključne točke se onda prosljeđuju klasifikatoru gesti kako bi napravio predikciju. Klasifikator gesti je ostvaren kao model umjetne neuronske mreže, izrađen pomoću biblioteke TensorFlow. Za treniranje i testiranje modela korišteni su postojeći skupovi podataka s Kaggle. Prikazana je programska implementacija u Pythonu i dobiveni rezultati.

Abstract (english)

This paper covers the development of a system for recognizing the letters of the American Sign Language alphabet in real time, which includes the detection of the hand on the camera and the making of and use of a gesture classifier. Hand detection is implemented using the MediaPipe library. MediaPipe draws a simplified skeleton of the hand on the screen and detects hand key points. These key points are then passed to the gesture classifier in order to make a prediction. The gesture classifier was realized as a model of an artificial neural network, created using the TensorFlow library. Existing Kaggle datasets were used to train and test the model. This paper shows the implementation of this system in Python and the obtained results.

• Keywords: prepoznavanje gesti ruku; umjetna inteligencija; računalni vid; neuronske mreže; američki znakovni jezik; Python; OpenCV; MediaPipe; TensorFlow
• Keywords (english): hand gesture recognition; artificial intelligence; computer vision; neural networks; American Sign Language; Python; OpenCV; MediaPipe; TensorFlow
• Language: croatian
• URN:NBN: urn:nbn:hr:168:766208
• Study programme: Title: Computing; Study programme type: university; Study level: undergraduate; Academic / professional title: sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva (sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-15 11:26:09