Tehnologije za prikupljanje, obradu i analizu velikih skupova podataka

master's thesis

• Title (english): Technologies for collecting, processing and analyzing large data sets
• Author: Nika Požar
• Mentor: Boris Milašinović (mentor)
• Committee member: Vedran Mornar (predsjednik povjerenstva)
• Committee member: Igor Mekterović (član povjerenstva)
• Committee member: Nikica Hlupić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2023-07-07, Croatia

Abstract

Kao jedna od modernih problematika je rad s velikim skupovima podataka. Ovaj rad upoznaje čitatelja s tom problematikom te iznosi tehnološka rješenja koja su se razvila za rad s tokom velikih skupova podataka. Detaljnije se opisuje primjer toka podataka od Spotify API-ja čiji se podaci povlače pomoću Apache Kafka tehnologije, preko Azure oblaka i transformacija pomoću Apache Sparka i Apache Iceberga do Microsoft Power BI alata za vizualizaciju. Koristeći primjer, objašnjavaju se prednosti i mane modernih tehnologija i naglašava važnost rada s velikim skupovima podataka.

Abstract (english)

One of the modern issues is working with large data sets. This paper introduces the reader with this problem and presents technological solutions that have been developed for working with the flow of large data sets. It presents a detailed example of a data flow from the Spotify API whose data is pulled using Apache Kafka technology, through the Azure cloud and transformed using Apache Spark and Apache Iceberg to the Microsoft Power BI visualization tool. Using an example, the advantages and disadvantages of modern technologies are explained and the importance of working with large data sets is emphasized.

• Keywords: veliki skupovi podataka; dohvaćanje podataka; transformacija podataka; vizualizacija podataka; Spotify API; Apache Kafka; Azure Blob Storage; Apache Spark; Apache Iceberg; Microsoft Power BI
• Keywords (english): big data sets; data retrieval; data transformation; data visualization; Spotify API; Apache Kafka; Azure Blob Storage; Apache Spark; Apache Iceberg; Microsoft Power BI
• Language: croatian
• URN:NBN: urn:nbn:hr:168:184918
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-07-13 10:29:49