Improved Human-Robot Interfaces for Urban Search and Rescue


A view of UML's current USAR interface. An augmented interface increases the user's situation awareness when operating a robot remotely.

Situation awareness (SA) is critical to the successful operation of an unmanned vehicle. Since 2003, we have studied numerous robot control systems intended for emergency response. We have designed evaluation methods and produced design guidelines for HRI. From this, we implemented our own multi-phase video-centric USAR system. Our platform is an iRobot ATRV-Jr featuring a 2.8GHz Pentium IV processor, a full sonar ring (26 sonars), two Canon VC-C4 pan-tilt-zoom cameras (front and rear), a forward- facing SICK laser, a forward-facing FLIR Systems Indigo A10 camera, a carbon dioxide sensor, an audio speaker and microphone, and a lighting system. The robot currently uses four levels of autonomy, ranging from tele-operation to a goal-centric mode.

Our interface utilizes many novel features such as the fusion of directsensor information to increase SA. Specifically, we combine laser and sonar information to indicate obstacles within a one-meter radius of the robot (shown directly below main video display) and map the environment during operation (shown upper left of main video display). We also fuse FLIR with video (shown in the main video display), which allows the operator to easily notice potential unseen victims.

Other innovative features are included in our implementation. On the main video display, a cross-hair indicates the current orientation of forward camera. A small mirrored video display simulates a rear-view mirror in the upper right. We have the ability to swap front and rear camera views; the drive controls and distance information are remapped to reflect this change. This virtual exchange eliminates the need for the robot to backup, which results in more efficient operation.

Related Papers

Brenden Keyes, Mark Micire, Jill L. Drury and Holly A. Yanco. Improving Human-Robot Interaction through Interface Evolution. Human-Robot Interaction, Daisuke Chugo (Ed.), INTECH, 2010.

Mark Micire, Jill L. Drury, Brenden Keyes, and Holly Yanco. Multi-Touch Interaction for Robot Control. International Conference on Intelligent User Interfaces (IUI), Sanibel Island, Florida, February 8–11, 2009.

Mark Micire, Jill L. Drury, Brenden Keyes, Holly Yanco, and Amanda Courtemanche. Performance of Multi-Touch Table Interaction and Physically Situated Robot Agents.  Poster presentation at the 3rd ACM/IEEE International Conference on Human-Robot Interaction, Amsterdam, March 2008.

Amanda Courtemanche, Mark Micire and Holly Yanco. Human-Robot Interaction using a Multi-Touch Display. 2nd Annual IEEE Tabletop Workshop, poster presentation, October 2007.

Mark Micire, Martin Schedlbauer and Holly Yanco. Horizontal Selection: An Evaluation of a Digital Tabletop Input Device. In Proceedings of the 13th Americas Conference on Information Systems, Keystone, CO. August 2007.

Holly A. Yanco and Jill L. Drury. Rescuing Interfaces: A Multi-Year Study of Human-Robot Interaction at the AAAI Robot Rescue Competition. Autonomous Robots, Volume 22, Number 4, pp. 333-352. May 2007. 

Brenden Keyes. MS Thesis: Evolution of a Telepresence Robot Interface. University of Massachusetts Lowell, Lowell, MA. April 2007.

Jill L. Drury, Brenden Keyes and Holly A. Yanco. LASSOing HRI: Analyzing Situation Awareness in Map-Centric and Video-Centric Interfaces. Proceedings of the Second Annual ACM/IEEE Conference on Human-Robot Interaction, March 2007.

Holly A. Yanco, Michael Baker, Robert Casey, Brenden Keyes, Philip Thoren, Jill L. Drury, Douglas Few, Curtis Nielsen and David Bruemmer. Analysis of Human-Robot Interaction for Urban Search and Rescue. Proceedings of the IEEE International Workshop on Safety, Security and Rescue Robotics. National Institute of Standards and Technology (NIST), Gaithersburg, MD, August 22-24, 2006.

Brenden Keyes, Robert Casey, Holly A. Yanco, Bruce A. Maxwell, and Yavor Georgiev. Camera Placement and Multi-Camera Fusion for Remote Robot Operation. Proceedings of the IEEE International Workshop on Safety, Security and Rescue Robotics. National Institute of Standards and Technology (NIST), Gaithersburg, MD, August 22-24, 2006.

Michael Baker. MS Thesis: An Intelligent Suggestion System for Improved Human-Robot Interaction. University of Massachusetts Lowell, Lowell, MA. July 2006.

 Michael Baker, Robert Casey, Brenden Keyes and Holly A. Yanco. Improved Interfaces for Human-Robot Interaction in Urban Search and Rescue. Proceedings of the IEEE Conference on Systems, Man and Cybernetics, October 2004. Nominated for best student paper award.

Michael Baker and Holly A. Yanco. Autonomy Mode Suggestions for Improving Human-Robot Interaction. Proceedings of the IEEE Conference on Systems, Man and Cybernetics, October 2004.

Dan Hestand and Holly A. Yanco. Layered Sensor Modalities for Improved Feature Detection. Proceedings of the IEEE Conference on Systems, Man and Cybernetics, October 2004.

Holly A. Yanco and Jill Drury. "Where Am I?" Acquiring Situation Awareness Using a Remote Robot Platform. Proceedings of the IEEE Conference on Systems, Man and Cybernetics, October 2004.