Raspodijeljeni modeli prostorno-vremenski ponderirane i grupirane regresije korištenjem platforme Apache Spark

master's thesis

• Title (english): Distributed Models of Geo-Temporal Weighted and Clustered Regression using Apache Spark
• Author: Darko Britvec
• Mentor: Krešimir Pripužić (mentor)
• Committee member: Krešimir Pripužić (predsjednik povjerenstva)
• Committee member: Siniša Sovilj (član povjerenstva)
• Committee member: Jurica Babić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2021-07-09, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Analiza i ekstrakcija znanja iz velikih prostorno-vremenskih skupova podataka sve su više zastupljeni kako u istraživačkim radovima tako i u stvarnim programskim rješenjima. U ovom radu analizirani su modeli strojnog učenja koji iskorištavaju postojanje prostorne i vremenske heterogenosti u podacima. U radu je implementiran raspodijeljeni model prostorno-vremenski ponderirane regresije (GTWR) korištenjem platforme Apache Spark. Arhitektura tog modela ne omogućava efektivnu stvarnovremensku obradu velikih skupova podataka u računalnom grozdu. Stoga je u radu predložen te implementiran raspodijeljeni model prostorno-vremenski grupirane regresije (GTCR) koji umjesto postupka ponderiranja koristi algoritam grupiranja kako bi opisao prostornu i vremensku nestacionarnost podataka. Naposljetku, modeli su isprobani i uspoređeni na stvarnom skupu podataka koji opisuje koncentracije najučestalijih onečišćivača zraka sa 12 mjernih postaja u Pekingu kroz period od 2013. do 2017. godine.

Abstract (english)

Analysis and data mining performed on big spatio-temporal datasets is more and more frequent topic both in research and the software development. Machine learning models analysed in this paper take advantage of the spatio-temporal heterogeneity in the data. The paper covers the implementation of the Geo-Temporally Weighted Regression (GTWR) distributed machine learning model using the Apache Spark platform. This model does not support the effective near-real-time computation in cluster. Geo-Temporally Clustered Regression (GTCR) model was introduced in this paper to accommodate the need of the machine learning model which can describe the spatio-temporal non-stationarity in large datasets. Finally, models are tested and compared using the real-world dataset which consists of measurements of most common air pollutants from 12 stations in Beijing in a period from 2013 to 2017.

• Keywords: prostorno-vremenska analiza; prostorno-vremenski ponderirana regresija; prostorno-vremenski grupirana regresija; raspodijeljena obrada velikih skupova podataka; Apache Spark
• Keywords (english): spatio-temporal analysis; Geo-Temporally Weighted Regression; Geo-Temporally Clustered Regression; distributed big data computing; Apache Spark
• Language: croatian
• URN:NBN: urn:nbn:hr:168:496608
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-13 11:53:26