Prepoznavanje teksta generiranog velikim jezičnim modelom

master's thesis

• Title (english): Detection of Text Generated by a Large Language Model
• Author: Maria Fain
• Mentor: Goran Delač (mentor)
• Committee member: Goran Delač (predsjednik povjerenstva)
• Committee member: Marin Šilić (član povjerenstva)
• Committee member: Klemo Vladimir (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2024-07-04, Croatia

Abstract

Veliki jezični modeli koriste se za probleme iz područja obrade prirodnog teksta poput klasifikacije, generiranja, sažimanja teksta i slično. Zlouporabom generativnih velikih jezičnih modela stvorila se potreba za sustavima sposobnima prepoznati tekst generiran velikim jezičnim modelom među tekstovima koje su napisali ljudi. Za rješavanje ovog problema programski su ostvareni klasifikatori BERT, RoBERTa i DistilBERT naučeni koristeći skup podataka „GPT Wiki Intro”. Dio tekstova u skupu za učenje generiran je velikim jezičnim modelom Curie. Nakon što su klasifikatori istrenirani, provedeno je vrednovanje uspješnosti njihovih predikcija nad skupom podataka za ispitivanje koristeći razne metrike. Klasifikatori su dodatno vrednovani koristeći tekstove koje su generirali različiti veliki jezični modeli.

Abstract (english)

Large language models (LLMs) are used for problems in the field of natural language processing such as text classification, generating or summarizing text and others. The misuse of generative LLMs created a need for systems which would be capable of detecting texts generated by an LLM among texts written by humans. For solving this problem, classifiers BERT, RoBERTa and DistilBERT were implemented and trained using the "GPT Wiki Intro" dataset. A share of the texts in the training dataset was generated by the Curie LLM. After the classifiers were trained, their predictions on the testing dataset were validated using various metrics. The classifiers were additionally validated using texts that were generated by different LLMs.

• Keywords: obrada prirodnog jezika; veliki jezični modeli; klasifikacija teksta; BERT; RoBERTa; DistilBERT; modeli zasnovani na transformerima; matrica zabune
• Keywords (english): natural language processing; large language models; text classification; BERT; RoBERTa; DistilBERT; transformer models; confusion matrix
• Language: croatian
• URN:NBN: urn:nbn:hr:168:792042
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2024-10-12 20:33:36