Computer Science ETDs
Publication Date
7-1-2014
Abstract
Robots are often highly non-linear dynamical systems with many degrees of freedom, making solving motion problems computationally challenging. One solution has been reinforcement learning (RL), which learns through experimentation to automatically perform the near-optimal motions that complete a task. However, high-dimensional problems and task formulation often prove challenging for RL. We address these problems with PrEference Appraisal Reinforcement Learning (PEARL), which solves Preference Balancing Tasks (PBTs). PBTs define a problem as a set of preferences that the system must balance to achieve a goal. The method is appropriate for acceleration-controlled systems with continuous state-space and either discrete or continuous action spaces with unknown system dynamics. We show that PEARL learns a sub-optimal policy on a subset of states and actions, and transfers the policy to the expanded domain to produce a more refined plan on a class of robotic problems. We establish convergence to task goal conditions, and even when preconditions are not verifiable, show that this is a valuable method to use before other more expensive approaches. Evaluation is done on several robotic problems, such as Aerial Cargo Delivery, Multi-Agent Pursuit, Rendezvous, and Inverted Flying Pendulum both in simulation and experimentally. Additionally, PEARL is leveraged outside of robotics as an array sorting agent. The results demonstrate high accuracy and fast learning times on a large set of practical applications.
Language
English
Keywords
Reinforcement learning, Motion planning, Robotics, Artificial Intelligence, Unmanned Aerial Vehcile, Systems control
Document Type
Dissertation
Degree Name
Computer Science
Level of Degree
Doctoral
Department Name
Department of Computer Science
First Committee Member (Chair)
Estrada, Trilce
Second Committee Member
Fierro, Rafael
Third Committee Member
Moses, Melanie
Fourth Committee Member
Williams, Lance
Project Sponsors
Sandia National Laboratories and New Mexico Space Grant
Recommended Citation
Faust, Aleksandra. "Reinforcement Learning and Planning for Preference Balancing Tasks." (2014). https://digitalrepository.unm.edu/cs_etds/43
