Robotic Task Learning

Faculty Members

Dr. Lydia Tapia

Postdocs

Dr. Satomi Sugaya

Grad Students

Torin Adamson
John Baxter
Mohammad R. Yousefi

Alumni

Dr. Nick Malone
Dr. Aleksandra Faust

Collaborators

Dr. Andrew R. Ferdinand
Dr. John Wood
Dr. Brandon Rohrer
Dr. Ron Lumia

Related Projects

Adaptive Quadrotor Control

Robotic task learning is a challenging and emerging area of research. Tasks are any discrete goal which a robot must accomplish. However, it is not yet clear how to consistently teach a robot how to perform even the simplest tasks which a human child can perform with ease. The canonical task: "Bring me that glass of water" is beyond the capabilities of the most advanced learning algorithms. Robotic learning uses a wide variety of machine learning techniques ranging from reinforcement learning to demonstration learning. Our research is focused on reinforcement learning (RL) and ways in which to optimize the convergence time of RL algorithms.

Reinforcement Learning is a special area of machine learning that is focused on maximizing rewards obtained by the agent. We utilize a RL package called BECCA (https://github.com/brohrer/becca), which was developed by Brandon Rohrer, previously at Sandia National Laboratories.


Block Diagram of the component of BECCA

BECCA is a general reinforcement learning agent tied to a feature creator. The learning agent keeps a model of cause-effect pairs and choses the next action based on matching the current working memory to the cause-effect pairs. The working memory is simply a short history of previous observations and actions. The feature creator builds up groups of related inputs and sends these inputs to the reinforcement learner. More about BECCA can be learned at Brandon's website here (http://brohrer.github.io/).

Reinforcement Learning struggles with large state spaces, so we have combined Reinforcement Learning with Probabilistic Roadmap Methods (PRMs). PRMs sample configurations in the environment and then connect configurations in a neighborhood to form a roadmap. This roadmap provides an approximated model of state space. The size of the state space is one of the largest factors in convergence time of a Reinforcement Learning algorithm, and thus the approximated model provided by the PRM aids greatly in this regard.


Example 2DoF Roadmap

Comparison of State Space Scaling

Using this PRM-BECCA learning scheme we can have BECCA learn to navigate a 7DoF WAM robot in pointing tasks. The interesting change this has on the problem is that the learning algorithm is largely agnostic to the DoFs and the number of nodes in the roadmap, in terms of iterations required to converge. However, increasing the number of nodes in the roadmap does increase the time it takes to process each iteration. This means that the learning algorithm is bounded by the roadmap size needed to solve a particular problem instead of by the complexity of the robot.


Cumluative Reward for Varying DoFs

Cumluative Reward for Varying Nodes in the Roadmap

Videos

BECCA Pointing Task on a WAM

Publications & Papers

  • John Baxter, Torin Adamson, Satomi Sugaya, Lydia Tapia, "Exploring Learning for Intercepting Projectiles with a Robot-Held Stick", In Proceedings of the IEEE Internatioanl Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, Sept. 2021.(pdf,video)

  • Satomi Sugaya, Mohammed R. Youssefi, Andrew R. Ferdinand, Lydia Tapia, "Multitask and Transfer Learning of Geometric Robot Motion", In Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, Sept. 2021.(pdf)

  • Nick Malone, Aleksandra Faust, Brandon Rohrer, Ron Lumia, John Wood, Lydia Tapia, "Efficient Motion-based Task Learning for a Serial Link Manipulator", In Transactions on Control and Mechanical Systems, 3(1), Jan. 2014. (pdf, Bibtex)

  • Nick Malone, Aleksandra Faust, Brandon Rohrer, John Wood, Lydia Tapia, "Efficient Motion-based Task Learning," Robot Motion Planning Workshop, In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vilamoura, Portugal, Oct. 2012.
    (pdf, BibTex, abstract, presentation)

  • Nick Malone, Brandon Rohrer, Lydia Tapia, Ron Lumia, John Wood, "Implementation of an Embodied General Reinforcement Learner on a Serial Link Manipulator," In Proc. IEEE International Conference on Robotics and Automation (ICRA), pp. 862-869, St. Paul, MN, U.S.A., May 2012.
    (pdf, BibTex, abstract, presentation)