Apstraktivno sažimanje teksta pomoću dubokog učenja

master's thesis

• Title (english): Abstractive Text Summarization Using Deep Learning
• Author: Ivan Anđelković
• Mentor: Marin Šilić (mentor)
• Committee member: Marin Šilić (predsjednik povjerenstva)
• Committee member: Siniša Srbljić (član povjerenstva)
• Committee member: Goran Delač (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2022-07-01, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Apstraktivno sažimanje jedno je od dva smjera u strojnom sažimanju teksta, uz ekstraktivno. Arhitekture modela koji se koriste pri rješavanju zadataka u ovom području bazirane su na ideji transformera koji se sastoji od para enkoder-dekoder. Trenutno najuspješniji modeli bazirani su na nenadziranom predtreniranju na velikim skupovima podataka i daljnjem podešavanju parametara na manjim skupovima podataka korištenjem nadziranog učenja. U radu su testirani i validirani upravo tri takva modela za apstraktivno sažimanje teksta: BERT, BART i BRIO te su evaluirani pomoću ROUGE metrika. Cilj je izvući bit iz teksta smanjujući njegovu veličinu. Za pravilnu generaciju sažetaka potrebno je poznavanje cijelog jezika kako bi se mogle odabrati prikladne riječi iz rječnika. Trenutni modeli su uspješni u stvaranju sažetaka koji su sadržajno isti ili suženi u odnosu na izvorni tekst. Razumiju značenje ulaznog teksta i koriste ga kao referencu pri generiranju novog kraćeg teksta. Za rad u području apstraktivnog sažimanja teksta potrebne su velike količine vremena i računalne snage što i dalje čini ovakve zadatke izazovnima.

Abstract (english)

Abstractive Text Summarization is one of two ways to automatically summarize text, along with extractive. The model architectures used in solving tasks in this area are based on the idea of a transformer consisting of an encoder-decoder pair. Currently, the most successful models are based on unsupervised pre-training on large datasets and further parameter tuning on smaller datasets using supervised learning. In this paper, exactly three such models for abstract text compression were tested and validated: BERT, BART and BRIO and were evaluated using ROUGE metrics. The goal is to get the meaning out of a body of text while also reducing it in size. Proper generation of summaries requires knowledge of the entire language in order to be able to select appropriate words from the dictionary. Current models are successful in creating summaries that keep the meaning the same or narrowed compared to the original text. They understand the meaning of the input text and use it as a reference when generating new, shorter text. Working in the field of abstract text summarization requires large amounts of time and computing power, which still makes such tasks challenging.

• Keywords: BART; BERT; BRIO; Pažnja; LSTM; ROUGE; Huggingface; Transformer; CNN Daily Mail skup; Apstraktivno sažimanje; Pred treniranje; Duboko učenje
• Keywords (english): BART; BERT; BRIO; Attention; LSTM; ROUGE; Huggingface; Transformer; CNN Daily Mail Dataset; Abstractive Summarization; Pre training; Deep learning
• Language: croatian
• URN:NBN: urn:nbn:hr:168:457879
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-14 08:18:20