Obrada velikih podataka korištenjem grafičkih procesora na platformi Apache Spark

master's thesis

• Title (english): Big Data Processing using Graphical Processors on Apache Spark
• Author: Darija Lukić
• Mentor: Krešimir Pripužić (mentor)
• Committee member: Krešimir Pripužić (predsjednik povjerenstva)
• Committee member: Marin Vuković (član povjerenstva)
• Committee member: Dragan Jevtić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2022-07-12, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Apache Spark je programski okvir za obradu velikih skupova podataka koji obradu može obavljati na jednom ili više računala. Nudi razvojne API-je za nekoliko programskih jezika od kojih je u studijskom slučaju ovog rada korištena Java. Programske naredbe iz dostupnih razvojnih API-ja izvršavaju se na procesorskoj jedinici - procesoru. Procesor je dizajniran za brzo rješavanje širokog raspona zadataka, ali je ograničen u istodobnosti zadataka koji se mogu izvoditi. Masivnu istodobnost izvršavanja zadataka nudi grafička procesorska jedinica - grafički procesor. Tvrtka NVIDIA razvila je akcelerator RAPIDS namijenjen za grafičke procesore proizvođača NVIDIA. Uključivanjem akceleratora u Sparkove programe koji koriste DataFrame API, određene programske naredbe iz tog API-ja, za koje je podrška razvijena, prebacuju svoje izvođenje s procesora na grafički procesor. Time se paralelizira i ubrzava obrada podataka. U ovom radu je teorijski opisana platforma Apache Spark i akcelerator RAPIDS, a njihova uporaba prikazana je u studijskom slučaju koji obrađuje podatke iz izvora podataka koji je Apache Kafka tema.

Abstract (english)

Apache Spark is a software framework for processing large data sets that can be processed on one or more computers. It provides high-level APIs for several programming languages, of which Java was used in the case study of this paper. Program commands from available development APIs are executed on the central processing unit - processor. Processor is designed to quickly solve a wide range of tasks, but is limited in the concurrency of tasks that can be performed. Massive simultaneity of task execution is offered by a graphics processing unit - a graphics processor. NVIDIA has developed the RAPIDS accelerator for NVIDIA graphics processors. By including accelerator in programs that use the Spark DataFrame API, certain program commands from that API, for which support has been developed, switch their performance from CPU to GPU. This parallelizes and speeds up data processing. This paper theoretically describes the Apache Spark platform and the RAPIDS accelerator, and their use is presented in a case study that processes data from a data source that is an Apache Kafka topic.

• Keywords: Apache Spark; centralna procesorska jedinica; grafička procesorska jedinica; RAPIDS akcelerator; Apache Kafka
• Keywords (english): Apache Spark; central processing unit; graphics processing unit; RAPIDS accelerator; Apache Kafka
• Language: croatian
• URN:NBN: urn:nbn:hr:168:367169
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-14 09:19:32