Distribuirana arhitektura za kratkoročne vremenske prognoze

master's thesis

• Title (english): Distributed architecture for short-term weather forecast
• Author: Jurica Kenda
• Mentor: Vinko Lešić (mentor)
• Committee member: Vinko Lešić (član povjerenstva)
• Committee member: Anita Banjac (član povjerenstva)
• Committee member: Branimir Novoselnik (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2023-06-28, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Ovaj rad zaranja duboko u arhitekturu i implementaciju infrastrukture za sustav kratkoročne vremenske prognoze. Njegova primarna svrha je prikazati različite inženjerske i proizvodne ustupke tijekom izgradnje takvog sustava, kao i pružiti primjenjiv sustav za proizvodnu upotrebu.
Dokument razlaže početne zahtjeve dionika i opisuje životni ciklus sustava od dizajna do detalja implementacije. Među mnogim različitim zahtjevima, onaj na koji se ovaj rad najviše fokusira je jednostavna integracija klijenta na oba kraja sustava. Koristi REST API za integraciju s proizvođačima informacija, Kafka redove poruka za prijenos poruka i Websockets za isporuku podataka klijentima. Autori koriste empirijsko stručno znanje u kombinaciji s temeljitim istraživanjem kako bi donijeli utemeljene odluke o tome kako će sustav funkcionirati i ispuniti svoje zahtjeve.
Konačni proizvod je infrastruktura visoke propusnosti napisana u jeziku Go, prenosi više od 300 poruka/sekundi, podržava više od 40 istodobnih klijenata (20 proizvođača, 20 klijenata) i postiže prosječnu latenciju od 100 milisekundi za prijenos poruka s kraja na kraj bez ispada poruka. Integracija sa sustavom zahtijeva <20 linija koda u bilo kojem modernom programskom jeziku (Python, Javascript, Java, Go…).

Abstract (english)

This paper dives deep into the architecture and implementation of an infrastructure for a nowcasting system. Its primary purpose is to display the various engineering and product trade offs while building such a system, as well as to provide an applicable system for a production use.
The paper breaks down the initial stakeholder requirements and describes the system lifecycle from the design to the details of implementation. Among many different requirements, the one this paper focuses on the most is seamless client integration on both ends of the system. It uses REST API for integrating with the information producers, Kafka message queues for message networking and Websockets for data delivery to the consuming clients. The authors use empirical expert knowledge, in combination with thorough research to make well-informed decisions about how the system will function and fulfil its requirements.
The end product is a high-throughput infrastructure written in Go, streaming over 300 messages/second, supporting over 40 concurrent clients (20 producing, 20 consuming), and achieving an average latency of 100 milliseconds for end-to-end message transfer with no message drops. Integrating with the system requires <20 LoC in any modern programming language (Python, Javascript, Java, Go…).

• Keywords: kratkoročna vremenska prognoza; infrastruktura; dizajn sustava; obrada podataka u stvarnom vremenu; REST; Kafka; Websockets
• Keywords (english): nowcasting; infrastructure; system design; real-time data processing; REST API; Kafka; Websockets
• Language: english
• URN:NBN: urn:nbn:hr:168:322788
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-10-11 09:17:25