Metode otkrivanja pomaka koncepta u tokovima podataka

master's thesis

• Title (english): Concept drift detection methods in data streams
• Author: Ela Grga
• Mentor: Alan Jović (mentor)
• Committee member: Alan Jović (predsjednik povjerenstva)
• Committee member: Marko Čupić (član povjerenstva)
• Committee member: Marin Golub (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2023-07-04, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Pojava tokova podataka uvela je novu dimenziju strojnog učenja. U tokovima podataka
podaci najčešće pristižu kontinuirano i velikom brzinom. Analiza toka podataka u stvarnom
vremenu zahtijeva algoritme i modele koji se mogu prilagoditi promjenjivoj distribuciji
podataka, nositi se s pomicanjem koncepta, neuravnoteženosti klasa i stvarati predviđanja u
hodu, kako podaci pristižu.
Cilj ovog diplomskog rada bio je usporediti dvije metode koje se bave pomakom koncepta
u tokovima podataka uz prisustvo neuravnoteženosti klasa. Metoda DDM-OCI u svrhu
pomaka koncepta u neuravnoteženom toku podataka prati odziv manjinske klase. Metoda
CIDD uravnotežava skup za učenje metodom ADASYN, a prati točnost modela metodom
ADWIN.
Ispitivanjem metoda nad jednim umjetnim i dva skupa podataka iz stvarnog svijeta, pokazalo
se da je metoda DDM-OCI veće točnosti u 9 od 10 ispitnih slučajeva i boljeg odziva
manjinske klase u 8 od 10 skupova ispitnih slučajeva u odnosu na metodu CIDD. Međutim,
ona to postiže uz cijenu čestog ponovnog učenja modela i s tim velikog trošenja resursa za
prikupljanje podataka svaki put kada je potrebno učenje modela

Abstract (english)

The emergence of data streams has introduced a new dimension to machine learning. In data
streams, data usually arrives continuously and at high speed. Real-time data stream analysis
requires algorithms and models that can adapt to changing data distributions, deal with
concept drift, class imbalances, and make predictions on the fly, as data arrives.
The aim of this master thesis was to compare two methods that deal with concept drift in
data streams in the presence of class imbalance. The DDM-OCI method follows the recall
of the minority class for the purpose of detecting concept drift in an imbalanced data stream.
The CIDD method balances the learning set using the ADASYN method, and monitors the
model accuracy using the ADWIN method.
By testing the methods on an artificial and two real-world datasets, it was shown that the
DDM-OCI method is more accurate in 9 out of 10 test cases and has a better response of the
minority class in 8 out of 10 test cases than the CIDD method. However, this was acheived
at the cost of frequent relearning of the model and, thus, there is a large expenditure of data
collection resources each time the model needs to be retrained.

• Keywords: tokovi podataka; strojno učenje; pomak koncepta; neuravnoteženost klasa; metode otkrivanja pomaka
• Keywords (english): data streams; machine learning; concept drift; class imbalance; drift detection methods
• Language: croatian
• URN:NBN: urn:nbn:hr:168:664624
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-10-11 08:57:08