Filtriranje geoprostornog toka podataka korištenjem platforme Apache Storm

master's thesis

• Title (english): Filtering of Geospatial Data Streams using Apache Storm Platform
• Author: Borna Bezjak
• Mentor: Krešimir Pripužić (mentor)
• Committee member: Krešimir Pripužić (predsjednik povjerenstva)
• Committee member: Alen Bažant (član povjerenstva)
• Committee member: Jurica Babić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2020-07-08, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Cilj rada je istražiti platformu Apache Storm i biblioteke Java Topology Suite, GeoTools i GeoSpark te pomoću navedenih tehnologija implementirati sustav za obradu geoprostornih podataka. U radu je opisana implementacija sustava pomoću arhitekture „objavi-pretplati“ koja obrađuje tokove dvodimenzionalnih geoprostornih podataka. Za dobavljanje ulaznih podataka i vraćanje rezultata koristi se neovisni Apache Kafka proizvođač i potrošač. Prilikom provođenja eksperimenata promatra se utjecaj strategija particioniranja prostora, broj korištenih particija i strategija indeksiranja podataka u određenoj particiji na vrijeme potrebno za obradu jednog podatka (tupla). Na temelju eksperimenata doneseni su zaključci koja kombinacija parametara se pokazala kao najoptimalnijom za obradu dvodimenzionalnih geoprostornih podataka.

Abstract (english)

The goal of the paper is researching platform Apache Storm, libraries Java Topology Suite, GeoTools i GeoSpark and by using mentioned technologies implementing application for filtering geospatial data. Paper describes implementation of system for processing two-dimensional geospatial data by using architecture “publish-subscribe “. Independent Apache Kafka producer and consumer are used for fetching data to application and for returning results back to user. Experiments are configured by three main parameters: chosen partitioning strategy, number of partitions and indexing strategy. During experimentations focus of observations are impacts these three parameters on time necessary to process single tuple. Results of experiments are evaluated and based on these results best possible strategy for processing two-dimensional geospatial data is determined.

• Keywords: tokovi podataka; geoprostorni podatci; Apache Storm; Apache Kafka; Java Topology Suite; GeoTools; GeoSpark; računalni grozd; particioniranje; particije; indeksiranje; topologija; eksperiment; optimalna strategija; prosječna obrada podatka
• Keywords (english): data streams; geospatial data; Apache Storm; Apache Kafka; Java Topology Suite; GeoTools; GeoSpark; cluster; computing; partitioning; partitions; indexing; topology; experiment; optimal strategy; average tuple processing time
• Language: croatian
• URN:NBN: urn:nbn:hr:168:847732
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-13 09:47:00