Jezični model temeljen na neuronskoj mreži

master's thesis

• Title (english): Neural language model
• Author: Florijan Stamenković
• Mentor: Marin Golub (mentor)
• Committee member: Marin Golub (predsjednik povjerenstva)
• Committee member: Željka Mihajlović (član povjerenstva)
• Committee member: Marko Čupić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2015-07-14, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

U ovom diplomskom radu razmatraju se četiri različita pristupa izgradnji jezičnih modela. U uvodu se daje kratak osvrt na moguće primjene jezičnih modela općenito. Potom se definiraju modeli bazirani na zaglađenom prebrojavanju n-grama (korištenjem aditivnog zaglađivanja i zaglađivanja Kneser-Ney metodom), umjetnoj neuronskoj mreži bez skrivenih slojeva, log-bilinearnom modelu i ograničenom Boltzmannovom stroju. Razmatraju se teoretske i implementacijske posebnosti opisanih modela. Potom se modeli vrednuju standardnim mjerama i po uspješnosti na zadatku Microsoft Research Sentence Completion. Dobiveni rezultati se uspoređuju s rezultatima iz sličnih radova. Daje se kratko razmatranje vremenske i memorijske složenosti izrade i korištenja modela, kao i usporedba vremena treniranja korištenjem središnjih i grafičkih procesora.

Abstract (english)

This master thesis cosiders four approaches to language model implementation. An introduction into language models and their application is presented. Following are the definitions of models based on n-gram counts with additive and Kneser-Ney smoothing, neural networks with no hidden layers, log-bilinear models and Restricted Boltzmann Machines. For each a theoretical description is given, as well implementation details. Models are evaluated using standard metrics and on Microsoft Research Sentence Completion Challenge. Results are compared to those from other papers. A brief comparison of model complexity (in time and memory space) is offered, as well as a comparison of iterative model training speed using central versus graphical processors.

• Keywords: jezični model; obrada prirodnog jezika; zaglađivanje \textit{n}-grama; Kneser-Ney; neuronska mreža; log-bilinear; Microsoft Research Sentence Completion Challenge; raspodjeljene reprezentacije; grafički procesor; Cuda
• Keywords (english): language model; natural language processing; n-gram smoothing; Kneser-Ney; neural networks; log-bilinear; Microsoft Research Sentence Completion Challenge; distributed representations; Word2Vec; GPU; Cuda
• Language: croatian
• URN:NBN: urn:nbn:hr:168:939585
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2018-02-06 12:05:09