Otkrivanje korisnih informacija iz životopisa zadanih u slobodnom obliku

master's thesis

• Title (english): Extraction of Relevant Information from Free-form Resumes
• Author: Davor Vukadin
• Mentor: Marin Šilić (mentor)
• Committee member: Marin Šilić (predsjednik povjerenstva)
• Committee member: Klemo Vladimir (član povjerenstva)
• Committee member: Goran Delač (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2020-07-07, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

U ovom radu prikazan je način rada arhitekture za obradu sekvencijalnih podataka - transformera te implementacije enkoderskog dijela u obliku jezičnog modela BERT. Implementirana su dva modela koja koriste BERT kao višejezični ekstraktor značajki te su učeni na dva skupa podataka, jedan za ekstrakciju "tvrdih" informacija i njihovog konteksta iz životopisa, a drugi za označavanje kvalitete samoprocjene korištenja vještina. Unutarnji slojevi pozornosti modela su analizirani kako bi se potvrdila važnost i funkcija novododanih tokena za konačnu klasifikaciju. Također su analizirane vektorske reprezentacije oba modela te njihov utjecaj na performanse.

Abstract (english)

This paper presents the operation of the architecture for sequential data processing - transformers and the implementation of the encoder part in the form of the language model BERT. Two models were implemented that use BERT as a multilingual feature extractor and were trained on two data sets, one to extract “hard” information and its context from a resume, and the other to label the quality of skill self-assessment. The inner attention layers of the model were analyzed to confirm the importance and function of the newly added tokens for the final classification. The embeddings of both models and their impact on performance were also analyzed.

• Keywords: duboko učenje; obrada prirodnog jezika; transformer; BERT; životopis; pozornost; višejezičnost; interpretabilnost
• Keywords (english): deep learning; natural language processing; transformer; BERT; resume; attention; multilingual; interpretability
• Language: croatian
• URN:NBN: urn:nbn:hr:168:361097
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-13 14:15:04