Title Quality of Experience estimation of encrypted video streaming by using machine learning methods Title (croatian) Procjena iskustvene kvalitete za strujanje šifriranoga videa primjenom metoda strojnoga učenja Author Irena Oršolić Mentor Lea Skorin-Kapov (mentor)Committee member Lea Skorin-Kapov (član povjerenstva)Granter University of Zagreb Defense date and country 2020, Croatia Scientific / art field, TECHNICAL SCIENCES Universal decimal classificationUDC ) 004 - Computer science and technology. Computing. Data processing Abstract With the amount of global network traffic steadily increasing, mainly due to video streaming services, network operators are faced with the challenge of efficiently managing their resources while meeting customer demands and expectations. A prerequisite for such Quality-of-Experience–driven (QoE) network traffic management is the monitoring and inference of application-level performance in terms of video Key Performance Indicators (KPIs) that directly influence end-user QoE. Given the persistent adoption of end-to-end encryption, operators lack direct insights into video quality metrics such as start-up delays, resolutions, or stalling events, which are needed to adequately estimate QoE and drive resource management decisions. This research has been motivated by the challenge to devise an approach for the estimation of QoE/KPIs from encrypted traffic, where we recognised machine learning (ML) methods as a promising way forward, and a fundamental part of the methodology. In this thesis, we present a generic methodology for training of ML–based models for the estimation of QoE and application-level KPIs of adaptive video streaming services, applicable in the context of traffic encryption. The methodology is embodied in the form of a conceptual framework which demonstrates the key methodological steps involved in developing an in-network QoE/KPI monitoring solution, including model training, model deployment, and model re-evaluation and adaptation. The methodology is evaluated through six studies, conducted over the course of four years, involving YouTube video on demand, resulting in models for session-level QoE/KPI estimation focused on both Android and iOS, models for near real-time KPI estimation, analysis of cross-platform and cross-network model applicability, analysis of methods for automated model re-evaluation and adaptation, and the analysis of the impact of the inclusion of application-level data (possibly shared by a service provider) on the performance of QoE/KPI estimation models. The key contribution of the thesis is a methodology that identifies relevant KPIs to be used as prediction targets, identifies relevant network traffic features obtainable on IP-level, and describes the procedures of model training, evaluation, re-evaluation, and adaptation, thus covering processes that are a prerequisite for actual model deployment. The practical focus of the thesis has been on YouTube, which is one of the most prominent video streaming services today. In that context, a unique and valuable contribution are also the models for YouTube QoE/KPI estimation focused on mobile platforms, applicable for both TCP and QUIC traffic, and employing a standardised ITU-T P.1203 model for the calculation of QoE. Moreover, the thesis presents models that include application-level context data as additional predictors, thus contributing to motivation for resolving existing issues standing in the way to QoE-centric cooperation among actors involved in the service delivery chain.
Abstract (croatian) Uslijed kontinuiranog rasta količine mrežnog prometa na globalnoj razini, čemu najviše doprinose usluge strujanja videa, mrežni operatori su suočeni s izazovom učinkovitog upravljanja mrežnim resursima, uz ispunjavanje zahtjeva i očekivanja krajnjih korisnika. Preduvjet za takvo upravljanje mrežnim prometom, vođeno iskustvenom kvalitetom (engl. Quality of Experience, QoE), je mogućnost praćenja i procjene performansi usluga strujanja videa na aplikacijskoj razini u obliku ključnih indikatora performansi (engl. Key Performance Indicator, KPI) koji direktno utječu na QoE krajnjih korisnika. Prijašnjih godina, rješenja za praćenje QoE-a u mreži su se oslanjala na inspekciju paketa za dobivanje informacija o kvaliteti videa, čitanjem podataka iz zaglavlja aplikacijske razine. Budući da je mrežni promet povezan s ovim uslugama sve češće šifriran s kraja na kraj, mrežni operatori više nemaju direktan uvid u mjere kvalitete strujanja, poput trajanja inicijalnog učitavanja (engl. initial delay, start-up delay), rezolucije videa ili trajanja zastoja u reprodukciji (engl. stalling, re-buffering), koje su nužne za adekvatnu procjenu QoE-a i upravljanje mrežnim resursima vođeno QoE-em. Ovo istraživanje je motivirano izazovom razvoja rješenja za procjenu QoE-a/KPI-eva iz šifriranog prometa, gdje smo prepoznali potencijal metoda strojnog učenja te su stoga temelj metodologije. U ovoj disertaciji prezentirana je generička metodologija za treniranje modela za procjenu QoE-a i aplikacijskih KPI-eva za usluge strujanja videa, temeljena na strojnom učenju, a primjenjiva u kontekstu šifriranog prometa. Metodologija je uobličena u konceptualni radni okvir kroz koji su demonstrirani ključni koraci razvoja rješenja za praćenje QoE-a/KPI-eva u mreži, što uključuje treniranje modela, ugradnju modela u mreži te re-evaluaciju i prilagodbu modela. Metodologija je iterativno unaprjeđivana i validirana kroz šest studija provedenih tokom perioda od četiri godine, čiji je praktični fokus usluga YouTube i strujanje videa na zahtjev (engl. Video on Demand). Šest provedenih studija opisanih u ovoj disertaciji rezultiralo je velikim brojem modela koji procjenjuju QoE/KPI-eve usluge YouTube u mreži na razini pojedinog videa ili na razini kratkih vremenskih intervala, koristeći pritom podatke prikupljene na Android i iOS platformi, u laboratorijskoj bežičnoj mreži i u mobilnoj mreži. Sa svakom studijom unaprijeđena je sveukupna metodologija, kako bi konačno poprimila oblik radnog okvira opisanog u ovoj disertaciji. Radni okvir i metodologija koju predstavlja su generički, a pojedinačne komponente radnog okvira su, u tom smislu, demonstrirane u praksi kroz slučajeve uporabe koji se tiču usluge YouTube.
Keywords
video streaming
Quality of Experience
QoE estimation
machine learning
Keywords (croatian)
video strujanje
iskustvena kvaliteta
procjena iskustvene kvalitete
strojno učenje
Language english URN:NBN urn:nbn:hr:168:474921 Project Number: UIP-2014-09-5605 Project Number: IP-2019-04-9793 Study programme Title: Computer Science Catalog URL http://lib.fer.hr/cgi-bin/koha/opac-detail.pl?biblionumber=52197 Type of resource Text Extent 189 str. File origin Born digital Access conditions Open access Terms of use Created on 2022-03-08 13:50:28
