Analiza velike količine podataka o stanju gradskog prometa upotrebom programskih alata otvorenog koda

master's thesis

• Title (english): City Traffic Big Data Analysis Using Open Source Software Solutions
• Author: Luka Kiseljak
• Mentor: Alan Jović (mentor)
• Committee member: Alan Jović (predsjednik povjerenstva)
• Committee member: Vlado Sruk (član povjerenstva)
• Committee member: Dejan Škvorc (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2020-07-13, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Porastom količina podataka koje programski sustavi prikupljaju i obrađuju, osobito početkom masovne upotrebe interneta, mnogi alati u upotrebi pokazali su se nedostatnima. Iz tog razloga, velik broj inženjera i znanstvenika uključio se u razvoj alata otvorenog koda namijenjenih upotrebi u novonastalom području velikih podataka. U ovom radu prikazana su tri takva alata, Kafka, Cassandra i Spark, proizvodi Apache Software Foundationa. Apache Kafka je alat za razmjenu poruka koji služi pojednostavljenju arhitekture sustava i predstavlja središnju točku preko koje se razmijenjuju svi podaci u sustavu. Apache Cassandra je raspodijeljena baza podataka koja omogućava čitanje i pisanje velikih količina podataka uz održavanje visokih performansi. Apache Spark je alat za raspodijeljenu obradu podataka poput napredne analitike ili upotrebe umjetne inteligencije. Na primjeru sustava koji analizira stanje u prometu demonstrirane su funkcionalnosti i upotreba navedenih alata u ostvarenju web-aplikacije koja korisnicima nudi pregled zapaženih prometnih poremećaja i predviđanje budućih.

Abstract (english)

With the dramatic increase in the quantities of data collected and processed by software systems, especially after massive adoption of internet, many tools in use have shown themselves as insufficiently capable. To address this issue, engineers and scientists from around the world have started building open source software tools intended for use in the newly established field of big data. In this thesis, three such tools are presented, Apache Software Foundation's Kafka, Cassandra and Spark. Apache Kafka is a messaging system which greatly simplifies big data system's architecture and presents a central information exchange point in these systems. Apache Cassandra is a distributed database which retains high performance despite data volumes stored in it. Apache Spark is a distributed tool for data processing such as advanced analytics and artificial intelligence usages. This thesis demonstrates features and use cases for these three tools using an example system for traffic data analysis, which exposes a web-aplication that allows users to browse collected traffic disruption data and find predictions for possible future such events.

• Keywords: Big data; Apache Kafka; Apache Cassandra; Apache Spark
• Keywords (english): Big data; Apache Kafka; Apache Cassandra; Apache Spark
• Language: croatian
• URN:NBN: urn:nbn:hr:168:702452
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-13 12:40:13