Podešavanje velikih jezičnih modela korištenjem tehnika poravnanja preferenci za identifikaciju investicijskih prilika

undergraduate thesis

• Title (english): Tuning large language models using preference alignment techniques to identify investment opportunities
• Author: Borna Josipović
• Mentor: Mario Brčić (mentor)
• Committee member: Mario Brčić (predsjednik povjerenstva)
• Committee member: Mihael Kovač (član povjerenstva)
• Committee member: Kristina Ćaleta (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2024-07-05, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Cilj ovog rada bio je istražiti metode podržanog učenja kako bi podesili veliki jezični model za prepoznavanje investicijskih prilika. Fokus je bio na dvije metode: izravno poravnavanje preferenci (DPO) i podržano učenje iz povratnih informacija ljudi (RLHF). Skup podataka izgrađen je koristeći vlastite scraper skripte te filtriranjem i procesiranjem članaka. Skup podataka u završnom obliku imao je oko 35 tisuća elemenata. Metoda DPO se pokazala kao jednostavno i učinkovito rješenje za naš problem, dok je metoda RLHF nepotrebno komplicirana i prezahtjevna. Rezultati su pokazali poboljšanje modela koristeći metodu DPO. Također je naglašena važnost prepoznavanja i upotrebe pravilne metode za specifični problem koji se rješava.

Abstract (english)

The goal of this bachelors thesis was to explore methods of reinforcement learning to fine-tune a large language model to recognize financial investment oppportunities. The focus was on two methods: direct preference optimisation (DPO) and reinforcement learning from human feedback (RLHF). The datawas was built using self-made scraper tools and methods of filtering and processing the articles. The final version of the dataset had around 35 thousand articles. DPO showed itself to be a very simple, useful and efficient method to implement, while RLHF turned out to be too complex and taxing. The results showed that the model was learning using the DPO method and that it is crucial to recognize and pick a method which aligns itself with our goals and problems.

• Keywords: Veliki jezični modeli; podržano učenje; izravno poravnavanje preferenci; podržano učenje iz povratne informacije ljudi; binarna klasifikacija; prepoznavanje investicijskih prilika
• Keywords (english): Large language models; reinforcement learning; direct preference optimisation; reinforcement learning from human feedback; binary classification; recognition of investment opportunities
• Language: croatian
• URN:NBN: urn:nbn:hr:168:258539
• Study programme: Title: Computing; Study programme type: university; Study level: undergraduate; Academic / professional title: sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva (sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2024-10-12 20:57:27