Prepoznavanje rukom pisanih matematičkih izraza pomoću konvolucijskih neuronskih mreža

master's thesis

• Title (english): Recognition of handwritten mathematical equations using convolutional neural networks
• Author: Danijel Stracenski
• Mentor: Marija Seder (mentor)
• Committee member: Marija Seder (član povjerenstva)
• Committee member: Ivan Petrović (član povjerenstva)
• Committee member: Juraj Peršić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2022-07-12, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

U ovom radu opisan je proces prepoznavanja i rješavanja rukom napisanih matematičkih izraza. Najprije su opisani svi koraci obrade ulazne slike izraza u željeni oblik. Nakon toga, opisan je algoritam detekcije pojedinog znaka unutar izraza koristeći iterativni DFS algoritam. Detektirani znakovi su zatim proslijeđeni klasifikacijskoj konvolucijskoj neuronskoj mreži čije su pojedinosti poput arhitekture, slojeva, funkcije gubitka i postupka učenja detaljno opisani. Kod klasifikacije je objašnjen i postupak kreiranja vlastitog seta rukom napisanih znakova. Klasificirani znakovi se konačno prosljeđuju modulu za stvaranje rekurzivnih izraza koristeći heuristike inspirirane matematičkim i gramatičkim pravilima. Na posljetku, opisana su grafička sučelja za unos i izračun rukom napisanih matematičkih izraza.

Abstract (english)

This thesis gives step by step explanation of the recognition and solving algorithms for handwritten math expressions. Firstly, image processing steps are explained in detail on the image containing the written expression. This is followed by details about the detection algorithm that finds symbols in expression image using an iterative DFS algorithm. Detected symbols are then passed to classifying convolutional network whose details like architecture, layers, loss function, and learning algorithm are described in detail. In the classification paragraph, there is also an explanation for the creation of a custom handwritten symbols dataset. Classified symbols are finally passed to a module designed for the creation of recursive math expressions using heuristics inspired by math and grammar rules. At the end of the thesis, there is a description of graphical user interfaces for writing and solving handwritten math expressions.

• Keywords: konvolucija; optičko prepoznavanje znakova; klasifikacija; segmentacija; heuristika; detekcija; matematički izraz; duboko učenje; strojno učenje; obrada slike; kompozit; rekurzivni izrazi; pytorch; ios
• Keywords (english): convolution; optical character recognition; classification; segmentation; heuristic; detection; math expression; deep learning; machine learning; image processing; composite; recursive expressions; pytorch; ios
• Language: croatian
• URN:NBN: urn:nbn:hr:168:821606
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-14 21:42:34