Generiranje stvarnovremenskih tokova senzorskih podataka parsiranjem izvora na webu

undergraduate thesis

• Title (english): Generating Real-Time Sensor Data Streams by Parsing Sources on the Web
• Author: Petar Gabrijel Kedmenec
• Mentor: Krešimir Pripužić (mentor)
• Committee member: Krešimir Pripužić (član povjerenstva)
• Committee member: Damjan Katušić (član povjerenstva)
• Committee member: Josip Marić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2021-07-09, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

U slučaju nedostatka Web sučelja koje bi davalo informacije o senzorskim očitanjima,
jedini način da dođemo do tih očitanja automatski je parsiranjem dokumenata u HTML
formatu koji sadrže listu tih očitanja. Stoga je u ovom radu je razvijena aplikacija za
parsiranje senzorskih podataka s Web-a korištenjem paketa HTTP Client, biblioteke Java
Topology Suite te biblioteke jsoup. Unutar postavki aplikacije se može birati vremenski
razmak između dva dohvaćanja i parsiranja HTML dokumenta, imena Kafka tema na koje
se šalju pojedini podaci te postaje čije podatke ćemo slati. Aplikacija svoju funkcionalnost
ostvaruje stvaranjem 3 dretve koje koristeći isti objekt Kafka klijenta na Apache Kafka
grozd šalju zapise s podacima koje su dobile parsiranjem datoteke u HTML formatu.

Abstract (english)

In the case of a lack of a Web API that would give us information about the readings of a
certain set of sensors, the only way to automatically obtain those reading is by parsing an
HTML document that contains a list of those readings. For this reason, in this thesis, an
aplication for parsing sensory information from the Web was created, using the HTTP
Client package, the Java Topology Suite library, and the jsoup library. Within the
aplication settings one can edit the period of time between two seperate fetchings and
parsings of an HTML document, the names of Kafka topics on which certain sets of data
are sent, as well as choosing which station's data we want to send. The application achieves
its funcionality by creating 3 threads, with each of the three sending records they got from
parsing the HTML document to the Apache Kafka cluster by using a single instance of a
Kafka client object.

• Keywords: Apache Kafka; jsoup; HTTP Client; HTML; parser; senzorski podaci
• Keywords (english): Apache Kafka; jsoup; HTTP Client; HTML; parser; sensor data
• Language: croatian
• URN:NBN: urn:nbn:hr:168:465938
• Study programme: Title: Computing; Study programme type: university; Study level: undergraduate; Academic / professional title: sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva (sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-14 20:45:38