Detekcija lažnih slika i video zapisa putem dubokog učenja

undergraduate thesis

• Title (english): Deep Learning Based Deepfakes Detection
• Author: Kristijan Grubešić
• Mentor: Vladimir Čeperić (mentor)
• Committee member: Vladimir Čeperić (predsjednik povjerenstva)
• Committee member: Ivan Grubišić (član povjerenstva)
• Committee member: Petra Bevandić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2020-09-02, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

U ovom radu dan je osnovan pregled pojma deepfake koji se pridaje svakoj slici ili videozapisu koji je modificiran pomoću strojnog i dubokog učenja najčešće s ciljem širenja dezinformacija. U drugom poglavlju opisan je proces stvaranja deepfake – ova pomoću autoencoding metode te raznih neuronskih mreža poput generativne suparničke mreže te je dan primjer programskog koda generativne suparničke mreže. U trećem poglavlju glavna tema je detekcija lažnih slika i videozapisa. Utvrđeno je kako je najbolja metoda primjena dubokog učenja s obzirom da ostale metode ne daju dobre rezultate. Pri detekciji lažnih slika koriste se konvolucijske neuronske mreže dok se pri detekciji lažnih videozapisa najčešće koriste metode vremenskih značajki ili metoda vizualnih artefakata pri čijoj se analizi koriste duboki ili plitki klasifikatori koji služe za ekstrahiranje artefakata iz videozapisa. U zadnjem poglavlju dan je pregled jednog od programa za detektiranje lažnih videozapisa koji za treniranje rabi FaceForensics++ set podataka koji je javno dostupan.

Abstract (english)

In this thesis a brief description of a term deepfake was given. Every image or a video that was modified using machine learning and deep learning with a goal of spreading misleading information, is called a deepfake. The process of creating a deepfake using autoencoding method and various types of neural networks, such as general adversial networks, was described in the second chapter. In the third chapter, main focus is put on detecting deepfakes, usually through machine learning, as it has proven to be the most efficient method that succeeds where others seems to fail. When detecting forged images, in most of the cases convolutional neural networks have been used. On the other hand, when detecting forged videos there's a focus on temporal features and visual artifacts both across and within each video frame. Also, deep and shallow classifiers have been used in order to extract visual artifacts from video frames. Finally, in the last chapter an example of a detecting program was given. Program was developed by company named Dessa and it mostly uses public FaceForensics++ dataset for training and does well on real world examples of deepfakes.

• Keywords: strojno učenje; duboko učenje; neuronske mreže; deepfake; autoencoding; generativne suparničke mreže; konvolucijske neuronske mreže; klasifikatori
• Keywords (english): machine learning; deep learning; neural networks; deepfake; autoencoding; generative adversial networks; convolutional neural networks; classifiers
• Language: croatian
• URN:NBN: urn:nbn:hr:168:496188
• Study programme: Title: Computing; Study programme type: university; Study level: undergraduate; Academic / professional title: sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva (sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-14 20:32:09