Korištenje velikih jezičnih modela u ocjenjivanju zadataka otvorenog tipa

undergraduate thesis

• Title (english): Using Large Language Models in evaluating open ended assignments
• Author: Lucija Runjić
• Mentor: Ljiljana Brkić (mentor)
• Committee member: Ljiljana Brkić (predsjednik povjerenstva)
• Committee member: Agneza Krajna (član povjerenstva)
• Committee member: Melita Fertalj (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2024-06-27, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

U ovom radu istražuje se primjena velikih jezičnih modela u ocjenjivanju zadataka otvorenog tipa. Konkretno, korišteni su modeli ChatGPT i Bard. Rad opisuje karakteristike velikih jezičnih modela te statističke mjere korištene za analizu, uključujući sumu kvadrata i koeficijent determinacije. Nadalje, prikazan je podatkovni skup korišten u istraživanju, s postupkom prikupljanja relevantnih podataka. Opisane su korištene tehnologije i alati koji su omogućili provedbu analize. Rad također nudi... More interpretaciju rezultata dobivenih analizom ocjenjivanja i povratnih informacija generiranih od strane velikih jezičnih modela. Uz to, predložena je mjera za ocjenjivanje sposobnosti vrednovanja zadataka ovog tipa, kao i implementacija algoritma za dodjeljivanje bodova temeljenih na kalibracijskom testu. Less

Abstract (english)

The purpose of this work is to determine the quality of large language models in evaluating open-ended assignments. More specifically, we are using ChatGPT and Bard. This paper describes characteristics of large language models in detail, along with various statistical measures used for analysis, such as sum of squares or coefficient of determination. Furthermore, the dataset used for the analysis is presented along with the process of acquiring and extracting all the relevant data from it.... More Technologies and other tools used analysis implementation are also described. The paper provides interpretation of the results of the analysis of the given dataset acquired through the grades and feedback generated by large language models. The measure for the evaluation the ability of grading tasks of this structure is proposed, along with the implementation of the algorithm for the distribution of points based on calibration test. Less

• Keywords: Veliki jezični modeli; ChatGPT; Bard; zadatci otvorenog tipa; suma kvadrata; koeficijent determinacije
• Keywords (english): Large language models; ChatGPT; Bard; open-ended assignments; sum of squares; coefficient of determination
• Language: croatian
• URN:NBN: urn:nbn:hr:168:562092
• Study programme: Title: Computing; Study programme type: university; Study level: undergraduate; Academic / professional title: sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva (sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Open access
• Created on: 2024-10-12 22:03:06