Prepoznavanje višealfabetnog teksta na slici korištenjem dubokog učenja

master's thesis

• Title (english): Multi-alphabet Text Recognition Using Deep Learning
• Author: Mladen Džida
• Mentor: Marin Šilić (mentor)
• Committee member: Marin Šilić (predsjednik povjerenstva)
• Committee member: Goran Delač (član povjerenstva)
• Committee member: Klemo Vladimir (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2023-07-04, Croatia

Abstract

U ovom radu je definirana arhitektura i provedena implementacija sustava za prepoznavanje višealfabetnog teksta na slici. Konkretno, opisani sustav je dizajniran za prepoznavanje latinice, arapskog i korejskog alfabeta. Sustav je implementiran u programskom jeziku Python, a za model je korišten vizualni transformer. Sa javnih internetskih stranica je prikupljen i pripremljen podatkovni skup za testiranje, a podatkovni skup za treniranje je umjetno generiran zbog nepostojanja dovoljno velikog javno dostupnog skupa. Model je, zbog ograničenih resursa, uspio samo u određenoj mjeri riješiti ovaj izazovni problem sa točnošću od 45.027%, ali glavni mu je doprinos to što predstavlja početnu točku za višealfabetno prepoznavanje teksta. Implementirani model ima mjesta za napredak, a najvažnija nadogradnja bi bila koristiti veći podatkovni skup za treniranje te isprobati veći skup metoda za pretprocesiranje slike. Trenirani su i modeli koji prepoznavaju zasebno svaki od tri navedena alfabeta. Modeli sa jednim alfabetom postižu bolje rezultate na tom individualnom alfabetu, ali se sami po sebi ne mogu koristiti kao sustav višealfabetnog prepoznavanja teksta jer im je potreban još jedan model koji će prvo odrediti o kojem se jeziku radi pa zatim proslijediti ulaznu sliku na odgovarajući model.

Abstract (english)

In this project, the architecture and the implementation of the system for recognizing multi-alphabet text on the image is carried out. In particular, the described system is designed to recognize Latin, Arabic and Korean alphabets. The system was implemented in the Python programming language, and a visual transformer was used for the model. A dataset for testing was collected and prepared from public websites, and a dataset for training was artificially generated due to the lack of a large enough publicly available datasets. The model, due to limited resources, only partially managed to solve this challenging problem with an accuracy of 45.027%, but its main contribution is that it represents a starting point for multi-alphabet text recognition. The implemented model has room for improvement, and the most important upgrades would be to use a larger training dataset and to try a larger set of image preprocessing methods. Models that recognize each of the three listed alphabets separately were also trained. Single-alphabet models perform better on that individual alphabet, but they by themselves cannot be used as a multi-alphabet text recognition system because they need another model to first determine what language is found on an image and then pass the input image to the appropriate model.

• Keywords: računalni vid; višealfabetno prepoznavanje teksta; duboko učenje; vizualni transformer; Python
• Keywords (english): computer vision; Multi-alphabet text recognition; deep learning; visual transformer; Python
• Language: croatian
• URN:NBN: urn:nbn:hr:168:677694
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-10-11 08:46:52