Human action and motion prediction in industrial human-robot shared environments using probabilistic decision-making methods

doctoral thesis

• Title (croatian): Predviđanje radnji i gibanja čovjeka u suradnim industrijskim prostorima ljudi i robota zasnovano na vjerojatnosnim metodama odlučivanja
• Author: Tomislav Petković
• Mentor: Ivan Marković (mentor)
• Committee member: Ivan Petrović (predsjednik povjerenstva)
• Committee member: Nikola Mišković (član povjerenstva)
• Committee member: Tamara Grujić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; (Department of Control and Computer Engineering); Zagreb
• Defense date and country: 2022-10-07, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing
• Universal decimal classification (UDC): 004 - Computer science and technology. Computing. Data processing

Abstract

As robots are progressing towards being ubiquitous and an indispensable part of our everyday environments efficient and safe collaboration and cohabitation become imperative. In this thesis, we focus on probabilistic decision-making models for human action and motion prediction in industrial human-robot shared environments. We first introduce a framework for human intention recognition in the robotized warehouse environment. This framework is based on Markov Decision Process action validation and the Hidden Markov Model intention recognition module. We continued by utilizing this framework for precise human motion prediction that served as input for the human-aware planning algorithm. We ran multiple experiments to test the proposed algorithms: in a real-world laboratory warehouse with a worker, wearing augmented reality glasses, a virtual reality warehouse twin as well an in-house developed warehouse simulator. Finally, we utilized Long Short-Term Memory networks to predict the next object human is going to pick in a collaborative environment. In order to reduce execution time, we crafted two dimensionality reduction methods. The first one is a feature selection method that relies on signal correlation and individual merit while the second one is a feature extraction method based on the autoencoder model. A method for human action prediction using a hidden Markov model for integrated warehouse systems based on human pose measurements and virtual reality data. A human motion prediction method for human-aware robot trajectory planning in integrated warehouse systems. A method for human action prediction based on their kinematic model and probabilistic decision-making in human-robot shared industrial environments.

Abstract (croatian)

S obzirom na njihov razvoj, roboti postaju sveprisutni i neizostavan dio našeg svakodnevnog okruženja te učinkovita i sigurna suradnja s čovjekom postaje imperativ. U ovoj disertaciji fokusiramo se na predviđanje radnji i gibanja čovjeka u suradnim industrijskim prostorima ljudi i robota. Prvo predstavljamo metode za prepoznavanje namjere čovjeka u integriranom robotiziranom skladištu. Predstavljena metoda se temelji na validaciji radnji čovjeka pomoću Markovljevog procesa odlučivanja i modulu za prepoznavanje namjere temeljenog na skrivenom Markovljevom modelu. Tu metodu dodatno koristimo za precizno predviđanje ljudskog kretanja koje je poslužilo kao ulaz za o čovjeku svjesno planiranje. Proveli smo više eksperimenata kako bismo testirali predložene algoritme: u stvarnom laboratorijskom skladištu s radnikom koji nosi naočale za proširenu stvarnost, digitalnom blizancu stvarnog skladišta te pomoću simulatora skladišta. Konačno, upotrijebili smo povratne mreže s dugom kratkoročnom memorijom kako bismo predvidjeli sljedeći objekt koji će čovjek dohvatiti. Kako bismo smanjili vrijeme izvršenja, koristili smo dvije metode smanjenja dimenzionalnosti. Prva je metoda odabira značajki koja se temelji na korelaciji signala, dok je druga metoda izdvajanja značajki temeljena na modelu autoenkodera. Metoda predviđanja čovjekova djelovanja pomoću skrivenog Markovljeva modela za integrirane skladišne sustave zasnovana na mjerenju položaja čovjeka i podacima iz virtualne stvarnosti. Metoda predviđanja čovjekova gibanja za planiranje trajektorija robota svjesno blizine čovjeka u integriranim skladišnim sustavima. Metoda predviđanja čovjekova djelovanja zasnovana na kinematičkom modelu čovjeka i vjerojatnosnom odlučivanju u suradnim prostorima ljudi i robota.

• Keywords: human action prediction; human motion prediction; Markov decision process; hidden Markov model; probabilistic decision-making methods; recurrent neural networks; long short-term memory networks; feature dimensionality reduction; autoencoders; collaborative environments
• Keywords (croatian): predviđanje čovjekova djelovanja; predviđanje čovjekova gibanja; Markovljevi procesi odlučivanja; skriveni Markovljev model; vjerojatnosni modeli odlučivanja; povratne neuronske mreže; autoenkoder; odabir značajki; povratna mreža s dugom kratkotrajnom memorijom; suradni prostori
• Language: english
• URN:NBN: urn:nbn:hr:168:261340
• Promotion: 2022
• Study programme: Title: Computer Science; Study programme type: university; Study level: postgraduate; Academic / professional title: Doktor znanosti (Doktor znanosti)
• Type of resource: Text
• Extent: 119 str. : graf. prikazi ; 30 cm. + CD-ROM.
• File origin: Born digital
• Access conditions: Open access
• Created on: 2022-11-28 09:59:20