Robotsko učenje za zadatke manipulacije

master's thesis

• Title (english): Robot learning for manipulation tasks
• Author: Marko Kuzmić
• Mentor: Marija Seder (mentor)
• Committee member: Ivan Marković (predsjednik povjerenstva)
• Committee member: Ivan Petrović (član povjerenstva)
• Committee member: Luka Petrović (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2024-07-10, Croatia

Abstract

Ovaj rad istražuje primjenu Q-learning algoritma za učenje zadataka robotske manipulacije u simuliranim okruženjima Mountain-Car i Cart-Pole. Q-learning je off-policy algoritam za učenje putem potkrepljenja (reinforcement learning) koji uči optimalne akcije temeljeći se na interakciji s okruženjem i dobivenim nagradama. Eksperimentalni rezultati pokazuju da je Q-learning algoritam uspješno naučio agente kako izvršavati složene zadatke manipulacije, postižući visoke performanse u oba okruženja. U Mountain-Car okruženju, agent je naučio iskoristiti inerciju automobila za savladavanje brda, dok je u Cart-Pole okruženju agent uspješno naučio balansirati polugu na kolicima. Prednosti Q-learning algoritma uključuju jednostavnost, robusnost i jamstvo konvergencije prema optimalnoj politici uz pravilno postavljanje hiperparametara. Međutim, algoritam ima ograničenja vezana uz diskretizaciju prostora stanja i memorijske zahtjeve. Budući rad predlaže primjenu deep Q-learninga, naprednih strategija istraživanja, optimizaciju hiperparametara, primjenu na stvarnim robotskim sustavima i istraživanje hibridnih pristupa kako bi se poboljšala učinkovitost i skalabilnost algoritma. Ovi rezultati pružaju osnovu za daljnja istraživanja i primjenu Q-learning algoritma u stvarnim robotskim sustavima.

Abstract (english)

This thesis explores the application of the Q-learning algorithm for learning robotic manipulation tasks in simulated environments, specifically Mountain-Car and Cart-Pole. Q-learning is an off-policy reinforcement learning algorithm that learns optimal actions based on interaction with the environment and received rewards. Experimental results demonstrate that the Q-learning algorithm successfully taught agents to perform complex manipulation tasks, achieving high performance in both environments. In the Mountain-Car environment, the agent learned to use the car's inertia to climb the hill, while in the Cart-Pole environment, the agent successfully learned to balance the pole on the cart. The advantages of the Q-learning algorithm include simplicity, robustness, and guaranteed convergence to the optimal policy with proper hyperparameter tuning. However, the algorithm has limitations related to the discretization of state spaces and memory requirements. Future work suggests applying deep Q-learning, advanced exploration strategies, hyperparameter optimization, application to real robotic systems, and exploring hybrid approaches to improve the efficiency and scalability of the algorithm. These results provide a basis for further research and application of the Q-learning algorithm in real robotic systems.

• Keywords: reinforcement learning (RL); robotska manipulacija; sintetički podaci; simulacije; autonomni robotski sustavi
• Keywords (english): reinforcement learning (RL); robot manipulation; synthetic data; simulations; autonomous robotic system
• Language: croatian
• URN:NBN: urn:nbn:hr:168:205283
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2024-10-12 21:17:02