Primjena prijenosnog učenja za prilagodbu metoda računalnog vida blisko-infracrvenim kamerama

master's thesis

• Title (english): Application of Transfer Learning for Adapting Computer Vision Methods to Near-Infrared Cameras
• Author: Mihael Kožul
• Mentor: Tomislav Petković (mentor)
• Committee member: Tomislav Petković (predsjednik povjerenstva)
• Committee member: Zvonko Kostanjčar (član povjerenstva)
• Committee member: Stjepan Begušić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2025-02-27, Croatia

Abstract

Praćenje ljudskih pokreta nalazi primjenu u raznim područjima poput biomehanike, sporta, virtualne stvartnosti i nadzora. U ovom radu istražuje se prilagodba modela OpenPose za analizu pokreta pomoću blisko infracrvenih (NIR) kamera. Standardni OpenPose model treniran je na RGB slikama, što može dovesti do smanjene preciznosti pri radu s NIR slikama. Da bi se to ispitalo, provodi se prijenosno učenje nad OpenPose modelom koristeći snimke dobivene NIR kamerama te RGB slike transformirane tako da oponašaju NIR spektar. Eksperimentalni dio uključuje usporedbu modela treniranog na RGB podacima, modela treniranog na stvarnim NIR podacima te modela treniranog na transformiranim RGB slikama. Transformacije uključuju uzimanje samo crvenog kanala RGB slike te korištenje težinske kombinacije kanala za NIR simulaciju. Rezultati pokazuju da model treniran na stvarnim NIR slikama postiže najbolje rezultate, no modeli s transformiranim podacima također ostvaruju solidne performanse. Model treniran težinskom kombinacijom RGB kanal pokazao se boljim od modela treniranog isključivo crvenim kanalom, ali nije mnogo lošiji od najboljeg modela, što sugerira njegovu primjenjivost kao alternativa ili dopuna NIR skupa podataka.

Abstract (english)

Tracking human movements finds application in various fields such as biomechanics, sports, virtual reality and surveillance. This paper explores the adaptation of the OpenPose model for motion analysis using near-infrared (NIR) cameras. The standard OpenPose model is trained on RGB images, which can lead to reduced accuracy when working with NIR images. To test this, transfer learning is performed on the OpenPose model using images obtained from NIR cameras and RGB images transformed to mimic the NIR spectrum. The experimental part includes a comparison of a model trained on RGB data, a model trained on real NIR data, and a model trained on transformed RGB images. The transformations include taking only the red channel of the RGB image and using a weighted combination of channels for the NIR simulation. The results show that the model trained on real NIR images achieves the best results, but models with transformed data also achieve solid performance. The model trained with the RGB channel weight combination performed better than the model trained exclusively with the red channel, but not much worse than the best model, suggesting its applicability as an alternative or supplement to the NIR data set.

• Keywords: prijenosno učenje; OpenPose; Caffe; PyTorch; NIR format; fino podešavanje
• Keywords (english): transfer learning; OpenPose; Caffe; PyTorch; NIR format; fine-tuning
• Language: croatian
• URN:NBN: urn:nbn:hr:168:599801
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Open access
• Created on: 2025-03-12 11:10:55