Automatska detekcija demencije iz govora koristeći transformerske modele

master's thesis

• Title (english): Automatic Detection of Dementia from Speech Using Transformer Models
• Author: Lovro Matošević
• Mentor: Alan Jović (mentor)
• Committee member: Alan Jović (predsjednik povjerenstva)
• Committee member: Ante Đerek (član povjerenstva)
• Committee member: Marin Golub (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2022-07-06, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Demencija je ozbiljna bolest koja je najčešće kronične ili progresivne naravi te je jedan od glavnih uzroka invalidnosti među starijom svjetskom populacijom. Automatska detekcija demencije težak je zadatak. Uključuje analizu akustičkih značajki govora, lingvističkih značajki transkripata i slično. Ovaj rad bavi se istraživanjem mogućnosti modela dubokog učenja koji iz lingvističkih transkripata te direktno iz govora detektiraju potencijalnu demenciju. Korišten je skup podataka Pittov korpus,... More dio DementiaBank dijeljene baze podataka čija je glavna namjena proučavanje demencije i srodnih bolesti. eksperimentirano je s modelima kao što su BERT, RoBERTa, XLNet i Audio Spectrogram Transformer. Također, istražene su mogućnosti modela Wav2Vec2 za automatsko prepoznavanje govora kojim su pripremljeni transkripti koje su zatim poslužile kao ulaz u tekstne modele. Pokazano je da se korištenjem kvalitetnih transkripata mogu postići točnosti detekcije iznad 90%. Konačno, predstavljena je detaljna diskusija rezultata, u kojoj su između ostaloga uspoređeni rezultati korištenih pristupa detekciji demencije, te je predstavljena jezgrovita analiza određenih pogreški.
Less

Abstract (english)

Dementia is a serious disease that is usually chronic and progressive, while also being one of the leading causes of disability among the elderly population. Automatic detection of dementia is a difficult task. It includes the analysis of acoustic features of speech, linguistic features of transcripts and other similar features. This work explores the capabilities of deep learning models to accurately detect dementia from linguistic transcripts or directly from speech. The Pitt corpus... More dataset was used, which is a part of DementiaBank, a shared database intended for studying dementia and other similar diseases. Experiments were conducted with deep learning models such as BERT, RoBERTa, XLNet and Audio Spectrogram Transformer. Furthermore, Wav2Vec2 model was used for preparing transcripts that were later used as the input for text-based models, and its capabilities were assessed. It was shown that dementia detection using well-prepared speech transcripts can achieve accuracy rates of above 90%. Finally, a discussion on the results of the used approaches is given, and a brief analysis of common errors is presented. Less

• Keywords: demencija; duboko učenje; Pittov korpus; transformer; RoBERTa; XLNet; Audio Spectrogram Transformer; automatsko prepoznavanje govora
• Keywords (english): dementia; deep learning; Pitt corpus; transformer; RoBERTa; XLNet; Audio Spectrogram Transformer; automatic speech recognition
• Language: croatian
• URN:NBN: urn:nbn:hr:168:052863
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-14 09:29:54