Accessibility is more than an access device. It is the ability of a person to understand and manipulate a device. Accessibility depends on a person's motor dexterity, cognitive capabilities, sensory impairments, behavioral skills, and social skills. This research focuses on providing methods for independent manipulation of unstructured environments utilizing a wheelchair mounted robotic arm. The target audience would be power wheelchair users who may additionally have cognitive impairments. It was hypothesized that a vision-based interface would be easier to use then a menu-based system. With greater levels of autonomy, less user input would be necessary for control. Thus, by explicitly designating the end goal of a "pick-and-place" activity of daily living, the end user population could be expanded to include persons with cognitive impairments.
- The user “zooms in” on the doorknob using progressive quartering. The red box indicates the selected region which contains the desired object.
Towards this end, we designed and implemented human-robot interfaces compatible with indirect (e.g. single switch scanning) and direct (e.g. touch screen and joystick) selection. We implemented an autonomous system for the Manus robot arm to reach towards the desired object. We evaluated the interfaces and system with able-bodied participants to provide a baseline and with end-users. We developed interface design guidelines and experimental design guidelines for human-robot interaction with assistive technology.
- The user selects the desired object indicating "I want that" with either a touch screen or mouse-emulating joystick. The fixed camera view selection is shown on the left. The moving camera view is shown center and right, using touch screen and joystick, respectively.
In our current work, we have integrated our visual interface and reaching algorithms with the our University of Central Florida collaborator's object recognition and grasping algorithms. A video of the end-to-end system can be found here.
