Tumačenje neuronskih modela za obradu prirodnoga jezika

undergraduate thesis

• Title (english): Interpreting Neural Models for Natural Language Processing
• Author: Livio Benčik
• Mentor: Jan Šnajder (mentor)
• Committee member: Jan Šnajder (predsjednik povjerenstva)
• Committee member: Domagoj Alagić (član povjerenstva)
• Committee member: Martin Tutek (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2019-07-12, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Neuronski modeli danas nalaze široku primjenu u obradi prirodnoga jezika te polučuju izvrsne rezultate na nizu zadataka analize teksta. Istovremeno, budući da nemaju eksplicitnih značajki koje bi bile izgrađene na temelju tekstnog ulaza, nedostatak ovih modela jest da su nerijetko teško tumačivi, tj. teško je objasniti klasifikacijsku odluku pojedinačnih primjera. U literaturi je predloženo niz postupaka koji ciljaju povećati tumačivost neuronskih modela, kao što su brisanje reprezentacija ili postupci temeljeni na gradijentu. Tema završnoga rada jesu postupci za tumačenje neuronskih modela u zadatcima obrade prirodnoga jezika. Proučiti postupke za tumačenje odluka neuronskih modela s naglaskom na one primjenjive u obradi prirodnoga jezika. Implementirati odnosno prilagoditi postojeće implementacije neuronskih modela za nekoliko tipičnih zadataka obrade prirodnog jezika, poput analize sentimenta, odgovaranja na pitanja i zaključivanja. Odabrati jedan ili dva standardna postupka za tumačenje odluka neuronskih modela, primjerice brisanje reprezentacije ili analizu gradijenta, te ih primijeniti na ove modele i zadatke. Načiniti usporedbu i sustavnu analizu postupaka za različite modele i zadatke. Radu priložiti izvorni kod razvijenog sustava, označene skupove podataka, naučene modele te citirati korištenu literaturu.

Abstract (english)

Neural models are now being widely applied in natural language processing and yield excellent results on a wide range of text analysis tasks. At the same time, because they do not make use of explicit features derived from text, the models suffer from a lack of interpretability, i.e., often times the decision the the model makes on individual instances is hard to explain. A number of methods for increasing the interpretability of neural model has been proposed in the literature, such as representation erasure and gradient-based method. The topic of this thesis are the methods for interpreting neural models applied to natural language processing tasks. Do a literature survey on methods for interpreting the decisions of neural models, with an emphasis on methods that are applicable to natural language processing. Implement or adopt existing implementations of neural models for a few typical natural language processing tasks, such as sentiment analysis, question answering and natural language inference. Chose one or two standard methods for interpreting neural models, such as representation erasure and gradient-based methods, and apply them to these models and tasks. Do a comparison and a systematic analysis of the methods across different models and tasks. All references must be cited, and all source code, documentation, executables, and datasets must be provided with the thesis.

• Keywords: neuronski modeli; neuronske mreže; strojno učenje; obrada prirodnog jezika; interpretiranje; objašnjavanje; analiza osjetljivosti; slojevita propagacija značajnosti; brisanje reprezentacije; analiza sentimenta
• Keywords (english): neural models; neural networks; machine learning; natural language processing; interpreting; explaining; sensitivity analysis; layer-wise relevance propagation; representation erasure; sentiment analysis
• Language: english
• URN:NBN: urn:nbn:hr:168:458076
• Study programme: Title: Computing; Study programme type: university; Study level: undergraduate; Academic / professional title: sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva (sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2020-01-07 16:43:16