Prototip za primjenu modeliranja nestrukturiranih raspodijeljenih podataka s Interneta

undergraduate thesis

• Title (english): Prototype for the application of modeling unstructured distributed data from the Internet
• Author: Lucija Sarić
• Mentor: Mirta Baranović (mentor)
• Committee member: Mirta Baranović (predsjednik povjerenstva)
• Committee member: Slaven Zakošek (član povjerenstva)
• Committee member: Igor Mekterović (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2021-09-14, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Strukturirani podaci sadrže točno definirane tipove podataka s obrascima zbog kojih ih se lako pretražuje i analizira. Nestrukturirani podaci su svi podaci koji nisu strukturirani. Polustrukturirani podaci sadrže unutarnje oznake pomoću kojih se identificiraju odvojeni elementi podataka.
Strukturiranje podatka se vršilo uz RPA (Robotic Process Automation) i Eywa aplikaciju. Roboti su napravljeni koristeći Javascript uz Node.js okolinu i Selenium web pogonitelj. Njihova zadaća je prikupljanje podatka sa zadanih web stranica i pohranjivanje istih u MongoDB bazu podataka. Njihovim radom se nestrukturirani podaci strukturiraju u polustrukturirane podatke.
Model podataka se oblikovao u Eywa aplikaciji po njihovoj sintaksi. Prikazivao je pravila DnD-a pete edicije.
Uspješnom uspostavom modela podataka Eywa aplikacija generira PostgreSQL bazu podataka, potrebne relacije i GraphQL API za povezivanje s bazom i upravljanje podacima. Podatke koje su roboti pohranili u MongoDB bazu je potrebno izvesti (engl. export) u datoteke formata JSON. Zatim programska skripta čita svaku od tih datoteka, gradi potrebne GraphQL mutacije i korištenjem tih mutacija migrira podatke u PostgreSQL bazu. Time se polustrukturirani podaci strukturiraju u strukturirane podatke.
Korištenjem jezika Javascript (Node.js) i biblioteke React napravljen je prototip koji prikazuje dobivene strukturirane podatke. Prototip prikazuje sve zapise entiteta Race, Class, Monster, Item i Spell i vrijednosti svih njihovih atributa. Te zapise je moguće pretraživati i filtrirati po vrijednostima nekih njihovih atributa.

Abstract (english)

Structured data is compromised of cleary defined data types with patterns that make them easily searchable. Unstructured data is data that is not structured. Semi-structured data containes internalized labels that identify separate data elements.
Data structuring was done via RPA (Robotic Process Automation) and Eywa aplication. Robots were made using Javascript (Node.js) and Selenium web driver. Their task was data collecting from various web sites and storage of said data in MongoDB database. By processing data, robots structured unstructured data to semi-structured.
Data model was made using Eywa aplication using their syntax. Data model was made to represent rule of DnD 5th edition.
When data model is deployed, Eywa aplication generates PostgreSQL database, necessary relations and GraphQL API that is used to connect to the said database and manipulate data that is stored there. Data that robots stored in MongoDB database needs to be exported in JSON formated files. Python scrypt then reads these files, makes appropriate GraphQL mutations and using these mutations migrates data to generated PostgreSQL database. With that being done, semi-structured data becomes structured data.
Prototype was made using Javascript (Node.js) and React library. Said prototype shows structured data that was generated. It displays all records of Race, Class, Monster, Item and Spell entities. It also displays values of all of their attributes. These records can be searched and filtered using values of some of the attributes.

• Keywords: strukturirani podaci; nestrukturirani podaci; RPA
• Keywords (english): structured data; RPA; Robotic Process Automation; unstructured data
• Language: croatian
• URN:NBN: urn:nbn:hr:168:318631
• Study programme: Title: Computing; Study programme type: university; Study level: undergraduate; Academic / professional title: sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva (sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-13 13:46:20