Automatsko prepoznavanje znakovnog jezika

undergraduate thesis

• Title (english): Automatic sign language recognition
• Author: Luka Kurtin
• Mentor: Hrvoje Mlinarić (mentor)
• Committee member: Hrvoje Mlinarić (član povjerenstva)
• Committee member: Hana Ivandić (član povjerenstva)
• Committee member: Ana Šikić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2024-07-05, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Ovim radom detaljno je opisan postupak izrade mobilne aplikacije za uređaje sa sustavom Android kojom se omogućava prepoznavanje slova američke znakovne abecede. Za pripremu podataka i izradu modela napisani su programi u programskom jeziku Python koji koriste biblioteke NumPy, OpenCv, tkinter i Pandas te alat TensorFlow Lite i MediaPipe Hands radni ok. Program za izradu modela također koristi podatke dobivene iz programa za pripremu podataka spremljene u csv datoteci kao ulaz za model. Dobiveni model, zajedno s MediaPipe Hands radnim okvirom, koristi se za prepoznavanje znakovnog jezika u izrađenoj Android aplikaciji. Nakon objašnjenja Python programa, detaljno je opisan postupak izrade aplikacije, zajedno s prikazima i isječcima koda, te način na koji izrađena aplikacija prepoznaje znakovni jezik i sprema podatke u Room bazu podataka.

Abstract (english)

This paper provides a detailed description of the process of creating a mobile application for Android devices that enables the recognition of letters from the American Sign Language alphabet. For preparation of data and creation of the model, programs were written in the Python programming language using libraries such as NumPy, OpenCV, tkinter, and Pandas, as well as TensorFlow Lite tool and the MediaPipe Hands framework. The model creation program also utilizes data obtained from the data preparation program, saved in a CSV file, as input for the model. The resulting model, along with the MediaPipe Hands framework, is used for sign language recognition in the developed Android application. Following the explanation of the Python programs, the process of developing the application is described in detail, including screenshots and code snippets, and the way the application recognizes sign language and stores data in a Room database.

• Keywords: američki znakovni jezik; klasifikacija znakovnog jezika; MediaPipe Hands; TensorFlow Lite; Android Studio; programski jezik Kotlin; Jetpack Compose; Room baza podataka
• Keywords (english): American Sign Language; sign language classification; MediaPipe Hands; TensorFlow Lite; Android Studio; Kotlin programming language; Jetpack Compose; Room Database
• Language: croatian
• URN:NBN: urn:nbn:hr:168:398079
• Study programme: Title: Computing; Study programme type: university; Study level: undergraduate; Academic / professional title: sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva (sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Open access
• Created on: 2024-10-12 21:15:26