Decentralizirano upravljanje jatom robota korištenjem potpornog strojnog učenja

master's thesis

• Title (english): Decentralized control of a flock of robots using reinforcement machine learning
• Author: Lovro Gotovac
• Mentor: Stjepan Bogdan (mentor)
• Committee member: Stjepan Bogdan (predsjednik povjerenstva)
• Committee member: Zdenko Kovačić (član povjerenstva)
• Committee member: Goran Vasiljević (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2021-07-09, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Electrical Engineering

Abstract

Svrha ovog rada bila je, koristeći potporno učenje, naučiti jednog agenta (studenta) da se ponaša kao drugi agenti u jatu (učitelji) koji koriste Reynoldsova pravila. Odnosno, cilj je bio istrenirati kontroler temeljen na potpornom učenju koji se uči ``promatrajući'' izlaze iz Reynoldsovog kontrolera. U radu je dan kratki sažetak onih Reynoldsovih pravila korištenih za kontroler, kohezija, separacija, poravnanje i izbjegavanje prepreka. Također, dan je i sažetak potpornog učenja. Opisane su Bellmanove jednadžbe koje su i začele ideju o potpornom učenju. Opisan je Q-learning algoritam temeljen na tim jednadžbama. Objašnjen je razlog prelaska iz tabličnih algoritama kao što je Q-learning u aproksimativne algoritme kao što je DQN (aproksimacija Q vrijednosti) i DDPG (aproksimacija politike) te je dan njihov opis. Također, opisani su programski paketi korišteni u izradi ovog rada. Konačno, opisana je struktura koda korištenog za simulaciju i prikazani su rezultati učenja za sva tri algoritma. DDPG algoritam pokazao se najboljim za rješavanje ovakve vrste problema.

Abstract (english)

This thesis's main goal was to use reinforcement learning to train one agent (student) to behave like other agents in a flock (teachers), which use Reynolds rules to produce flocking. That is, the goal was to train an RL controller that learns by ``observing'' the outputs of the Reynolds controller. This thesis gives a short description of the main Reynolds rules used for the Reynolds controller, cohesion, separation, alignment, and obstacle avoidance, as well as a description and goals of reinforcement learning. First, the fundamental RL equations were described - the Bellman equations. Then, the Q-learning algorithm was explained and the need for value approximation (DQN) or policy approximation (DDPG) instead of the tabular methods. Finally, the structure of the used code was given, and the results of the three algorithms were analysed. It was shown that the DDPG algorithm performed best for this problem.

• Keywords: Reynoldsova pravila; potporno učenje; Q-learning; DQN; DDPG; ROS; Stage
• Keywords (english): Reynolds rules; reinforcement learning; Q-learning; DQN; DDPG; ROS; Stage
• Language: english
• URN:NBN: urn:nbn:hr:168:577957
• Study programme: Title: Electrical Engineering and Information Technology; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka elektrotehnike i informacijske tehnologije (magistar/magistra inženjer/inženjerka elektrotehnike i informacijske tehnologije)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-13 14:58:53