Procjena korisničkog ponašanja i performansi usluge YouTube na temelju analize kriptiranog mrežnog prometa

master's thesis

• Title (english): Estimation of User Interaction Behaviour and YouTube Performance Based on the Analysis of Encrypted Network Traffic
• Author: Ivan Bartolec
• Mentor: Lea Skorin-Kapov (mentor)
• Committee member: Lea Skorin-Kapov (predsjednik povjerenstva)
• Committee member: Ognjen Dobrijević (član povjerenstva)
• Committee member: Mirko Sužnjević (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2017-07-11, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Electrical Engineering

Abstract

Ovaj rad bavio se istraživanjem performansi YouTube-a u slučaju interakcije korisnika s iFrame video preglednikom implementiranom u prethodno implementiranoj YouQ Android aplikaciji. Tom su prilikom prethodno izgrađen YouQ sustav i metodologija izmijenjeni. U Android aplikaciju implementiran je snimatelj mrežnog prometa, skripte za obradu podatak su izmijenjene ili stvorene, aplikacija za računanje mrežnih atributa izmjenjena i prikupljen novi skup podataka za testiranje novih modela. Prvi dio rada opisuje korisničke scenarije te napravljene promjene u YouQ sustavu.

Svaki skup podataka napravljen je s ciljem raznih analiza na različitim razinama performansi YouTube-a. Prvo je analizirano kako korisnička interakcija utječe na aplikacijsku razinu. Zaključeno je kako pojedine interakcije uzrokuju razne događaje u video pregledniku (npr. pauza, zastoj) i kako uzrokuju kratke zastoje videa koji imaju izravan utjecaj na klasifikaciju iskustvene kvalitete. Analiza video spremnika pokazuje kako pojedine korisničke interakcije mogu biti jednostavno prepoznate i kako programsko sučelje vraća pogrešne rezultate prilikom pauziranja videa.

Sljedeće što je analizirano je mrežna razina sustava. Najbitniji dio analize je kako otkriti je li došlo do korisničke interakcije kako bi se novi model mogao primjeniti prilikom klasifikacije trenutnog videa. Analiza snimljenog mrežnog prometa u alatu Wireshark, koristeći razne funkcije poput IO grafova i statistika , pokazala je kako ne postoji pouzdan način otkrivanja korisničke interakcije u mrežnom prometu.

Posljednji dio rada je kreacija jednog velikog skupa podataka koji se sastoji od gotovo svih skupova podataka korištenih tijekom izrade ovog rada s gotovo 445 YouTube videa. Cilj ovog dijela rada kreiranje klasifikacijskog modela strojnog učenja korištenjem alata WEKA za pet različitih algoritama. Rezultati su pokazali kako novi model ima preciznost od 90%. Od svih obrađenih algoritama RandomForest ima najveću preciznost. Skup podataka obrađen je starom metodologijom i testiran starim YouQ modelom kako bi usporedili rezultate modela. Usporedba je pokazala kako se preciznost klasifikacije kreiranjem novog modela koji ukljućuje korisničke interakcije. Potrebno je napomenuti kako trenirajući skup podataka nije imao jednak broj klasificiranih videa. Iz tog razloga budući zadatak bit će prikupljanje dodatnog skupa podataka kako bi broj videa po klasama bio uravnotežen.

Abstract (english)

This thesis explored YouTube's performance when there was some user interaction being performed in the iFrame YouTube player of a previously developed YouQ Android application. To do so, most of the previously developed YouQ system was redeveloped and the test methodology has been adapted accordingly. In the Android application, network traffic capture was implemented, scripts for data processing have been changed and developed, application for calculating network features has been changed and new datasets were collected in a laboratory environment for testing the new models. The first part of this thesis focuses on used user interaction scenarios and newly implemented changes on the YouQ system.

A number of datasets were created in order to analyse the impact of user interaction on certain level of YouTube performance. The first thing that was analyzed is how user interaction impacts application-level data. What was concluded is that some interactions cause certain events (i.e., paused event, buffering event) in the player and produce stallings which have direct impact on the QoE classification. Analysis of buffer state shows how some user interactions can easily by detected and how in some cases the API for buffer information returns false information.

The next thing that was analysed is the network-layer data. Most interesting was how to detect user interaction in that layer so new prediction models could be applied on them. Analysis of captured network traffic was done in Wireshark with all its features such as IO graphs and conversation statistics. Results showed that there is no reliable way to detect user interaction from captured network traffic.

The final part of the thesis was creation of one large big dataset which consists of almost all datasets used throughout the thesis and of 445 YouTube videos. Main goal of this part was to create a machine learning classification model using the WEKA tool for five different algorithms. Results have shown how new model has an accuracy of around 90% with RandomForest being the algorithm with the highest accuracy. Dataset was also tested using the previously built YouQ system model (built without user interaction scenarios) to compare results. The comparison has shown that classification accuracy can be improved when building a new model involving user interaction scenarios. However, it should be noted that the training dataset did not have a balanced number of classified videos per class. For that reason, ongoing and future work will focus on collecting a larger and balanced dataset, and further validating developed models.

• Keywords: klasifikacija mrežnog prometa; strojno učenje; iskustvena kvaliteta; korisnička interakcija; YouTube
• Keywords (english): network traffic classification; machine learning; Quality of Experience; user interaction; YouTube
• Language: english
• URN:NBN: urn:nbn:hr:168:795925
• Study programme: Title: Information and Communication Technology; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka informacijske i komunikacijske tehnologije (magistar/magistra inženjer/inženjerka informacijske i komunikacijske tehnologije)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2019-03-13 18:53:59