Adaptive Motion Planning - Research Group
Home Research People Workshops Outreach Resources Videos    

Robotic Task Learning

Faculty Members
Dr. Lydia Tapia
Nick Malone
Aleksandra Faust
Dr. John Wood
Dr. Brandon Rohrer
Dr. Ron Lumia
Related Projects
Adaptive Quadrotor Control

Robotic task learning is a challenging and emerging area of research. Tasks are any discrete goal which a robot must accomplish. However, it is not yet clear how to consistently teach a robot how to perform even the simplest tasks which a human child can perform with ease. The canonical task: "Bring me that glass of water" is beyond the capabilities of the most advanced learning algorithms. Robotic learning uses a wide variety of machine learning techniques ranging from reinforcement learning to demonstration learning. Our research is focused on reinforcement learning (RL) and ways in which to optimize the convergence time of RL algorithms.

Reinforcement Learning is a special area of machine learning that is focused on maximizing rewards obtained by the agent. We utilize a RL package called BECCA (, which was developed by Brandon Rohrer, previously at Sandia National Laboratories.

BECCA is a general reinforcement learning agent tied to a feature creator. The learning agent keeps a model of cause-effect pairs and choses the next action based on matching the current working memory to the cause-effect pairs. The working memory is simply a short history of previous observations and actions. The feature creator builds up groups of related inputs and sends these inputs to the reinforcement learner. More about BECCA can be learned at Brandon's website here (").

Block Diagram of the component of BECCA

Reinforcement Learning struggles with large state spaces, so we have combined Reinforcement Learning with Probabilistic Roadmap Methods (PRMs). PRMs sample configurations in the environment and then connect configurations in a neighborhood to form a roadmap. This roadmap provides an approximated model of state space. The size of the state space is one of the largest factors in convergence time of a Reinforcement Learning algorithm, and thus the approximated model provided by the PRM aids greatly in this regard.

Example 2DoF Roadmap

Comparison of State Space Scaling

Using this PRM-BECCA learning scheme we can have BECCA learn to navigate a 7DoF WAM robot in pointing tasks. The interesting change this has on the problem is that the learning algorithm is largely agnostic to the DoFs and the number of nodes in the roadmap, in terms of iterations required to converge. However, increasing the number of nodes in the roadmap does increase the time it takes to process each iteration. This means that the learning algorithm is bounded by the roadmap size needed to solve a particular problem instead of by the complexity of the robot.

Cumluative Reward for Varying DoFs

Cumluative Reward for Varying Nodes in the Roadmap


Publications & Papers
  • Nick Malone, Aleksandra Faust, Brandon Rohrer, Ron Lumia, John Wood, Lydia Tapia, "Efficient Motion-based Task Learning for a Serial Link Manipulator," Transactions on Control and Mechanical Systems, Vol. 3, Num. 1, Janaury 2014. (pdf, Bibtex)

  • Nick Malone, Aleksandra Faust, Brandon Rohrer, John Wood, Lydia Tapia, "Efficient Motion-based Task Learning," Robot Motion Planning Workshop, In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vilamoura, Portugal, October 2012.
    (pdf, BibTex, abstract, presentation)

  • Nick Malone, Brandon Rohrer, Lydia Tapia, Ron Lumia, John Wood, "Implementation of an Embodied General Reinforcement Learner on a Serial Link Manipulator," In Proc. IEEE International Conference on Robotics and Automation (ICRA), pp. 862-869, St. Paul, Minnesota, May 2012.
    (pdf, BibTex, abstract, presentation)

Home | Research | People | Workshops | Outreach | Resources | Videos |  

Department of Computer Science,
1 University of New Mexico,
Albuquerque, NM 87131-0001 USA 
     Phone 505.277.3112     Fax 505.277.6927 

Department of Computer Science and Engineering | School of Engineering | University of New Mexico