Big Data tehnologije

undergraduate thesis

• Title (english): Big Data technologies
• Author: Ivan Dundović
• Mentor: Damir Pintar (mentor)
• Committee member: Damir Pintar (predsjednik povjerenstva)
• Committee member: Boris Vrdoljak (član povjerenstva)
• Committee member: Marko Banek (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2015-07-13, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Veliki podatci uključuju skupove podataka koji veličinom nadilaze mogućnosti uobičajeno korištenih računalnih alata da prikupe, upravljaju i procesiraju te podatke dovoljno brzo. Definiraju ih tri dimenzije: veličina, brzina i raznovrsnost. Relacijske baze podataka pokazale su se neprikladnim za rad s velikim podatcima. Nerelacijske baze podataka, s druge strane su skalabilnije, pokazuju bolje performanse i rješavaju mnoge probleme za čije rješavanje relacijske baze podataka nisu dizajnirane, poput upravljanja velikim, često nestrukturiranim, podacima. Za rad s velikim podatcima razvijene su mnoge tehnike, tehnologije i alati, među njima i Apache Hadoop. Apache Hadoop, ili kraće Hadoop, je radni okvir za distribuirano spremanje i procesiranje velikih količina podataka na spletu običnih računala kojih može biti od jednog do nekoliko tisuća. Hadoop datotečni sustav, YARN platforma za upravljanje resursima te MapReduce programski model, koji su u radu detaljno objašnjeni, čine osnovu Hadoopa. U petom poglavlju, koristeći Hadoop radni okvir na malom spletu računala, implementiran je MapReduce program koji sortira skup cijelih brojeva iz ulazne datoteke.

Abstract (english)

Big Data usually includes data sets whose size exceeds the capabilities of commonly used computer tools to capture, manage, and process data in a reasonable amount of time. Big Data is defined as three-dimensional, with those being volume, velocity, and variety. Relational databases are inadequate for processing such large data sets. Non-relational databases, on the other hand, are more scalable, show better performance and solve many problems that relational databases are not designed to solve. Many techiques, technologies, and tools were developed to manage Big Data, one of them being Apache Hadoop. Apache Hadoop, or just Hadoop, is framework for distributed storage and processing of large data sets on computer clusters composed of one to several thousand computers built from commodity hardware. Hadoop filesystem (HDFS), YARN platform for resource management and MapReduce programming model, which are covered in the paper in detail, form the base of Hadoop. In the fifth chapter, using Hadoop framework and MapReduce programming model on a small computer cluster, MapReduce software is implemented for sorting integer numbers that are read from input file.

• Keywords: Veliki podatci; Obrada podataka; SQL; Relacijski model baze podataka; NoSQL; Nerelacijski model baze podataka; Vertikalno skaliranje; Horizontalno skaliranje; Apache Hadoop; Distribuirani datotečni sustav; HDFS; MapReduce; YARN
• Keywords (english): Big Data; Data processing; SQL; Relational database model; NoSQL; Non-relational database model; Vertical scaling; Horizontal scaling; Apache Hadoop; Distributed filesystem; HDFS; MapReduce; YARN
• Language: croatian
• URN:NBN: urn:nbn:hr:168:102080
• Study programme: Title: Computing; Study programme type: university; Study level: undergraduate; Academic / professional title: sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva (sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2018-02-06 08:37:15