Adaptive Quadrotor Control

Unmanned aerial vehicles (UAVs) play an increasing role in a wide number of missions such as remote sensing, transportation, and search and rescue missions. Often, a critical part of a UAV's role is to carry loads vital to the mission. For example, cargoes may consist of food and supply delivery in disaster struck areas, patient transport, or spacecraft landing. Planning motions for a UAV with a load becomes difficult because load swing is difficult to control. However, it is a necessity for the safety and success of the mission.

We are interested in producing a trajectory for an aerial robot with a suspended load that delivers the load to a destination in a swing-free fashion while avoiding static obstacles.

We developed a motion planning framework for generating trajectories with minimal residual oscillations (swing-free). We rely on reinforcement learning algorithms to train the system for a particular load and to transfer the learned policy to different models and state and action spaces. Using this methodology, we are able to train the agent in two dimensional action space and fly a quadrotor in three dimensions, including ascending and descending. Integration of the Reinforcement learning with Probabilistic Roadmaps (PRMs) and path tracking algorithm allows us to create a trajectory with bounded load displacement that automatically avoids static obstacles.

Videos

Coffee Delivery

Aerial Cargo Delivery Through Reinforcement Learning

Learning Swing-free Trajectories for UAVs With Suspended Loads

Publications & Papers

  • Aleksandra Faust, Hao-Tien Chiang, Nathanael Rackley, Lydia Tapia, "Avoiding Moving Obstacles with Stochastic Hybrid Dynamics using PEARL:PrEference Appraisal Reinforcement Learning", In Proc. of IEEE International Conference on Robotics and Automation (ICRA), pp. 484-490, Stockholm, Sweeden, May 2016. (pdf, Bibtex)

  • Aleksandra Faust, Ivana Palunko, Patricio Cruz, Rafael Fierro, Lydia Tapia, "Aerial Suspendend Cargo Delivery through Reinforcement Learning", Artificial Intelligence Journal Special Issue on Learning and Robotic, 247, June 2017. (Bibtex)

  • Aleksandra Faust, Hao-Tien Chiang, Nathanael Rackley, Lydia Tapia, "Dynamic Obstacle Avoidance with PEARL: PrEference Appraisal Reinforcement Learning", In International Conference on Robotics and Automation (ICRA), pp. 484-490, Hamburg, Germany, Sept. 2015. (pdf, Bibtex)

  • Aleksandra Faust, Nick Malone, Lydia Tapia, "Preference-balancing motion planning under stochastic disturbances", In Proc. IEEE International Conference on Robotics and Automation (ICRA), page 3555-3562, Seattle, WA, U.S.A., May 2015. (pdf, Bibtex)

  • Aleksandra Faust, Nick Malone, Lydia Tapia, "Planning Preference-balancing Motions with Stochastic Disturbances", Machine Learning in Planning and Control of Robot Motion Workshop at IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Chicago, IL, U.S.A., Sep. 2014.
    (pdf, BibTex, abstract)

  • Aleksandra Faust, Peter Ruymgaart, Molly Salman, Rafael Fierro, Lydia Tapia, "Continuous Action Reinforcement Learning for Control-Affine Systems with Unknown Dynamics", Acta Automatica Sinica Special Issue of Extensions of Reinforcement Learning and Adaptive Control, 1(3) pp. 323-336, July 2014. Also, Technical Report TR13-002, Department of Computer Science, University of New Mexico, June 2014.
    (pdf, BibTex, abstract, Video) Technical Report( pdf)

  • Aleksandra Faust, Ivana Palunko, Patricio Cruz, Rafael Fierro, Lydia Tapia, "Aerial Suspended Cargo Delivery through Reinforcement Learning", Artifical Intelligence Journal 247 Special Issue on Leaning and Robotics, pp. 381-398, June 2015. Also, Technical Report TR13-001, Department of Computer Science, University of New Mexico, Aug. 2013.
    (pdf, abstract, Video)

  • Aleksandra Faust, Ivana Palunko, Patricio Cruz, Rafael Fierro, Lydia Tapia, "Learning Swing-free Trajectories for UAVs with a Suspended Load," IEEE International Conference on Robotics and Automation (ICRA), pp. 4887-4894, Karlsruhe, Germany, May 2013.
    (pdf, BibTex, abstract, Video, Matlab source code, presentation)

  • Ivana Palunko, Aleksandra Faust, Patricio Cruz, Lydia Tapia, Rafael Fierro, "A Reinforcement Learning Approach to Suspended Load Manipulation with Aerial Robots," IEEE International Conference on Robotics and Automation (ICRA), pp. 4881-4886, Karlsruhe, Germany, May 2013.
    (pdf, BibTex, abstract)

  • Rafael Figueroa, Aleksandra Faust, Patricio Cruz, Lydia Tapia, Rafael Fierro, "Reinforcement Learning for Balancing a Flying Inverted Pendulum", In Proc. The 11th World Congress on Intelligent Control and Automation (WCICA), pp. 3555-3562, Shenyang, China, July 2014.
    (pdf, BibTex, abstract, Video)

  • Aleksandra Faust, Ivana Palunko, Patricio Cruz, Rafael Fierro, Lydia Tapia, "Learning Swing-free Trajectories for UAVs with a Suspended Load in Obstacle-free Environments", Autonomous Learning workshop at IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany, May 2013. (pdf)