Electrical and Computer Engineering ETDs

Publication Date

Summer 7-3-2024

Abstract

In the future, city skies will be filled with Unmanned Aerial Vehicles (UAVs) for rapid urban transport, including parcel deliveries and air taxis. NASA's Urban Air Mobility (UAM) envisions UAVs navigating air corridors. These virtual pathways ensure safety and compliance with regulations. However, current research on UAM practical applications is limited. This dissertation focuses on designing air corridors, developing UAV control systems, and ensuring the robustness of control algorithms against disturbances in real-world environments.

Our design features an air corridor system with horizontal lanes and on-off ramps, conceptualized as cylindrical spaces and tori, respectively. To enable each UAV to locally optimize its acceleration based on its limited sensing area in a complex airspace, where some non-cooperative flying objects (NCFOs) may exist, we developed a novel deep reinforcement learning based control framework called Hybrid-Transformer Reinforcement Learning (HTransRL), leveraging Proximal Policy Optimization with Generalized Advantage Estimation to train a sophisticated model that decentralizes UAV control. This enables precise trajectory management, strict adherence to corridor boundaries, and effective collision avoidance. The hybrid-transformer architecture in HTransRL efficiently handles varying input sizes and types, compressing diverse observational data into a uniform format for processing by the deep actor-critic network. Additionally, HTransRL employs curriculum learning to enhance training efficiency.

When implementing HTransRL in real-world applications, it must adapt to disturbances like wind perturbations. To address this challenge, we implemented Federated Multi-agent Reinforcement Learning (FedMARL), applying federated learning to fine-tune HTransRL such that the model’s performance can be enhanced based on the current disturbances.

Keywords

reinforcement learing; Urban Air Mobility; air corridor; transformer; federated learning

Document Type

Dissertation

Language

English

Degree Name

Computer Engineering

Level of Degree

Doctoral

Department Name

Electrical and Computer Engineering

First Committee Member (Chair)

Xiang Sun

Second Committee Member

Rafael Fierro

Third Committee Member

Sudharman K. Jayaweera

Fourth Committee Member

Shuang Luan

Download

Share

COinS