Ulančavanje procesora za obradu toka podataka

master's thesis

• Title (english): Chaining Data Stream Processors
• Author: Ivana Mance
• Mentor: Krešimir Pripužić (mentor)
• Committee member: Krešimir Pripužić (predsjednik povjerenstva)
• Committee member: Martina Antonić (član povjerenstva)
• Committee member: Pavle Skočir (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2021-07-09, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Tokovi podataka su neprekinuti nizovi podataka koji nastaju u stvarnom vremenu. Za njihovu obradu razvijeni su posebni alati kao što je Apache Flink koji omogućava distribuirane transformacije i specifične korisničke procedure nad podacima, uz pohranu stanja. Iskorištavanjem Flinkove mogućnosti dijeljenog stanja, procesori su implementirani tako da se bez ponovnog pokretanja može mijenjati njihovo ponašanje. Konfiguracija procesora kompromis je između točnosti i latencije rezultata. Komunikacija između Flink procesora ostvaruje se čitanjem i pisanjem u Kafka teme. Rezultat je konfigurabilni sustav koji se u stvarnom vremenu može prilagođavati poslovnim zahtjevima.

Abstract (english)

Data streams are continuous sets of data generated in real time. To handle the volume of data streams, special tools have been developed such as Apache Flink, which enables distributed, stateful transformations of data, as well as specific user-defined procedures. By leveraging Flink's shared-state capabilities, processors are implemented in such a way that their behavior can be modified without stopping the execution. Processor configuration is a tradeoff between accuracy of the results and their latency. Communication between Flink processors is done by reading and writing in Kafka topics. The result is a configurable system that can be adapted to meet business requirements in real time.

• Keywords: distribuirana obrada; tokovi podataka; velike količine podataka; analiza; prepoznavanje anomalija
• Keywords (english): distributed processing; data streams; big data; data analysis; anomaly detection
• Language: croatian
• URN:NBN: urn:nbn:hr:168:975112
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-13 12:58:03