Pretraživanje hrvatskog leksičkog n-gramskog sustava na Hadoop platformi

undergraduate thesis

• Title (english): Searching Croatian lexical n-gram system on Hadoop platform
• Author: Petar Marić
• Mentor: Mirko Randić (mentor)
• Committee member: Šandor Dembitz (predsjednik povjerenstva)
• Committee member: Bruno Blašković (član povjerenstva)
• Committee member: Mirko Randić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2014-07-14, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Electrical Engineering

Abstract

Velike količine podataka zahtijevaju veće brzine obrade podataka, a jedno računalo ne može izvršavati takav posao u prihvatljivom vremenu. Radi toga je paralelna obrada podataka sve više u uporabi, a jedan od tih modela obrade podataka implementiran je u platformi Hadoop.
Platforma Hadoop sastoji se od nekoliko komponenti pri čemu svaka obavlja neki posao kao što je mapiranje blokova unutar Hadoop-ovog distribuiranog datotečnog sustava, raspodjela resursa po računalima unutar klastera, upravljanje pokrenutim aplikacijama, itd.
Kako hrvatski jezik ima veliku količinu riječi, zadatak - pronalaženje digrama i trigrama unutar velikih količina podataka, moguće je riješiti upotrebom Hadoop platforme što je vidljivo iz rezultata mjerenja prikazanih u ovom radu.

Abstract (english)

Large amounts of data require greater processing speed, and a computer can not perform such work in eligible time. Therefore the parallel processing increasingly in use, and one of these models of data processing is implemented in Hadoop platform.
Hadoop platform consists of several components whereby each performs a job such as mapping of blocks within Hadoop's distributed file system, the allocation of resources by the computers in the cluster, manage running applications, etc.
As Croatian language has a large amount of words, task - finding digrams and trigrams within large amounts of data, it is possible to solve using the Hadoop platform which is evident from results of the measurements presented in this work.

• Keywords: Hadoop; MapReduce; YARN; Mapper; Reducer; Java; Hrvatski leksički n-gramski sustav; Digrami
• Keywords (english): Hadoop; MapReduce; YARN; Mapper; Reducer; Java; Croatian lexical n-gram system; Digram
• Language: croatian
• URN:NBN: urn:nbn:hr:168:121849
• Study programme: Title: Computing; Study programme type: university; Study level: undergraduate; Academic / professional title: sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva (sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2018-02-05 17:43:04