Audio augmented reality as navigation aid for remotely operated vehicle piloting

doctoral thesis

• Title (croatian): Zvukom proširena stvarnost kao podrška u navigaciji daljinski upravljanim bespilotnim ronilicama
• Author: Antonio Vasilijević
• Mentor: Zoran Vukić (mentor)
• Committee member: Zoran Vukić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; (Department of Control and Computer Engineering); Zagreb
• Defense date and country: 2017, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Electrical Engineering; Automation and Robotics
• Universal decimal classification (UDC): 621.3 - Electrical engineering

Abstract

The majority of Human--Machine Interfaces (HMIs) designed for the tele-operation of unmanned vehicles only present information visually. Frequent overloading of the operator's visual channel may result in accidents. To reduce the occurrence of accidents and improve overall operating performance, this thesis proposes the implementation of the audio Augmented Reality (AR) paradigm in the form of a spatial auditory display for complex navigation tasks such as path following (route guidance) or trajectory tracking. Therefore, the first novelty of this thesis is the audio AR for human-in-the-loop systems that improves the efficiency of the operator in high visual workload scenarios. Another novelty of the thesis is development of the virtual target based guidance method for complex guidance tasks. The thesis proposes the application of a modified ``lookahead distance" guidance strategy, as a guidance law for generating a reference. The reference is presented to the operator as a spatial auditory image of the virtual target to be followed. The result is the second contribution of the thesis: dynamic virtual target guidance method suitable for complex guidance tasks of unmanned remotely operated marine vehicles. Before being presented to the operator, the reference information is preconditioned through ``supernormal" transformations to increase azimuth and distance resolution in regions of interest and thus to improve efficiency and spatial acuity, the comparative disadvantages of the hearing channel. The third contribution of the thesis is the reference remapping transformation, adjusted for human hearing, enhancing the spatial hearing resolution in the selected area of interest and improving guidance of remotely operated marine vehicles. A series of experiments show that the proposed guidance strategy provides a comprehensible and effective auditory reference. The enhanced supernormal auditory interface improves performance by reducing the tracking error of both path following and trajectory tracking compared to a non-enhanced interface without the supernormal transformation. Experimental data confirm that under high visual and cognitive load, auditory interface emerges as an important advantage. It was shown that especially in high-load operational environment unloading the operator yields improved navigation quality. Experiment participants report that the novel auditory interface is intuitive and easy to use, providing very good situational awareness. Presented results clearly show that there is a rationale for using AR paradigm (visual and audio) for the purpose of ROV navigation. While the auditory interface investigated in this paper is shown to be effective for the tele-operation of unmanned vehicles, it may have other applications suc

Abstract (croatian)

Većina sučelja čovjek-stroj (HMI), razvijenih za daljinsko upravljanje bespilotnim vozilima, prikazuje informacije isključivo vizualno. Preopterećenje vizualnog kanala operatora često može rezultirati nezgodama. Da bi se smanjila učestalost nezgoda i poboljšala ukupna operativna učinkovitost, ova disertacija predlaže primjenu paradigme zvukom proširene stvarnosti (AR), u obliku prostornog zvukovnog sučelja, za složene navigacijske zadaće kao što su praćenje putanje (path following) ili praćenje trajektorije (trajectory tracking). Stoga je prvi doprinos ove teze primjena paradigme zvukom proširene stvarnosti za sustave sa-čovjekom-u-petlji upravljanja, koja poboljšava učinkovitost operatera u zadaćama visokih vizualnih i kognitivnih opterećenja. Sljedeći doprinos teze je razvoj metoda daljinskog vođenja bespilotnih vozila (teleoperation) za složene navigacijske zadaće, temeljenih na virtualnim dinamičkim ciljevima. Predlaže se primjena modificirane strategije "gledišta na daljinu" (lookahead distance) za generiranje referenci. Referenca je prezentirana operatoru kao prostorna slušna slika virtualnog cilja koji treba slijediti, i metoda predstavlja drugi doprinos ovog rada. Prije prezentiranja operateru, referentni signal je transformiran korištenjem "nadnaravnih" (supernormal) transformacija. Cilj je poboljšati ljudsku slušnu razlučivost po azimutu i udaljenosti u područjima od interesa i time poboljšati učinkovitost i nadomjestiti komparativne nedostatke osjeta sluha. Stoga je treći doprinos teze „nadnaravna“ transformacija reference, prilagođene ljudskom sluhu, a koja povećava prostornu slušnu razlučivost u području od interesa i poboljšava vođenje daljinski upravljanih bespilotnih vozila. Niz eksperimenata pokazuje da predložena strategija pruža razumljivu i učinkovitu zvukovnu referencu. Poboljšano „nadnaravno“ zvukovno sučelje povećava učinkovitost obavljenih zadaća smanjujući pogrešku praćenja putanje i trajektorije u usporedbi s „normalnim“ sučeljem bez nadnaravne transformacije. Eksperimentalni podaci potvrđuju da pod visokim vizualnim i kognitivnim opterećenjem, zvukovno sučelje predstavlja važnu prednost i osigurava poboljšanu kvalitetu vođenja. Sudionici eksperimenta potvrđuju da je novo zvukovno sučelje intuitivno i jednostavno za korištenje, i da osigurava vrlo dobru prostornu orijentaciju. Prikazani rezultati jasno govore u prilog korištenju AR paradigme (vizualne i audio) za navigaciju bespilotnih vozila. Dok se zvukovno sučelje istraženo u ovome radu pokazalo učinkovitim za daljinsko upravljanje bespilotnim vozilima, ono sigurno može imati i druge primjene kao što su navigacija osoba oštećena vida ili u industriji računalnih igara.

• Keywords: Audio Augmented Reality; Human--Machine interface; Spatial Auditory Display; Virtual Target; Guidance; Tele-operation; Unmanned Vehicles; Supernormal Transformations
• Keywords (croatian): zvukom proširena stvarnost; sučelje čovjek-stroj; prostorno zvukovno sučelje; virtualni cilj; vođenje; bespilotna vozila; nadnaravna transformacija
• Language: english
• URN:NBN: urn:nbn:hr:168:036390
• Study programme: Title: Electrical Engineering and Computing; Study programme type: university; Study level: postgraduate; Academic / professional title: Doktor znanosti elektrotehnike i računarstva (Doktor znanosti elektrotehnike i računarstva)
• Type of resource: Text
• Extent: 140 str. ; 30 cm
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2019-04-05 12:54:26