Poopćivost tumačivih neuronskih modela obrade prirodnoga jezika

undergraduate thesis

• Title (english): Generalizability of Interpretable Neural Models for Natural Language Processing
• Author: Patrik Matošević
• Mentor: Jan Šnajder (mentor)
• Committee member: Jan Šnajder (predsjednik povjerenstva)
• Committee member: Domagoj Alagić (član povjerenstva)
• Committee member: Martin Tutek (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2020-07-06, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Duboki neuronski modeli korišteni u zadacima obrade prirodnog jezika s vremenom su
postali sve složeniji. To je dovelo do problema nemogućnosti tumačenja načina rada tih
modela, a uvođenjem mehanizma pozornosti ovaj je problem djelomično riješen. Mehanizam pozornosti mehanizam je odabira skrivenih stanja iz povratnih neuronskih mreža koji su važni za zadatak, a također često poboljšava generalizaciju modela. Zadatak ovog rada bio je testirati poboljšavaju li se generalizacijske performanse modela korištenje označenih obrazloženja u podacima. Razmatrana su tri modela: dva jednostavna bez mehanizma pažnje
i jedan s mehanizmom pažnje. Modeli su trenirani na tri skupa podataka s referentnog skupa
ERASER i njihova je izvedba analizirana pojedinačno na temelju toga jesu li se koristili podaci o obrazloženjima ili ne. Rezultati su pokazali da se podaci o obrazloženjima ne mogu koristiti da bi se poboljšali rezultati jednostavnijih modela baziranim na povratnim mrežama na načine koje smo uzeli u obzir. Međutim, uvođenjem mehanizma pažnje, podatke o obrazloženjima je moguće jednostavno i na prirodan način dodati u proces treniranja te je to poboljšalo performanse modela, ako je model bio prikladan za zadatak.

Abstract (english)

Deep neural models used in natural language processing tasks have become more complex over time. This led to a problem of the impossibility of interpreting the inner workings of the model, but this problem was partly solved after the introduction of the attention mechanism. The attention mechanism is a mechanism for selecting outputs from recurrent neural networks that are important for the task, and they often improve generalizability of the model. This thesis investigates whether incorporating information about human-annotated rationales improves the generalizability of the model. Three models are considered: two simple ones without attention mechanism, and one with the attention mechanism. Models were trained on three datasets from the ERASER benchmark and their performance was analyzed individually based on whether the rationale information was used or not. Results show that the rationale information cannot be used to improve the basic RNN models in the ways we considered. However, after adding the attention mechanism, the rationale information can be easily and naturally incorporated in the training process and, if the model is well suited for the task, this can improve the performance.

• Keywords: duboko učenje; obrada prirodnog jezika; povratne neuronske mreže; mehanizam pažnje; interpretabilnost
• Keywords (english): deep learning; natural language processing; recurrent neural networks; LSTM; attention mechanism; interpretability
• Language: english
• URN:NBN: urn:nbn:hr:168:621197
• Study programme: Title: Computing; Study programme type: university; Study level: undergraduate; Academic / professional title: sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva (sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-14 09:30:24