Virtual Tools: A Framework for Simplifying
Sensory-Motor Control in Complex Robotic Systems

Randal C. Nelson, Martin Jagersand, and Olac Fuentes
Department of Computer Science
University of Rochester
Rochester, NY 14627

Abstract: We describe and demonstrate a construct termed a ``virtual tool'', that provides a flexible interface to sensory-motor control. This interface is, from a user standpoint, substantially less complex, and more application-oriented than the raw devices. The basic idea is to use extra degrees of freedom present in a flexible system, in conjunction with sophisticated sensing (e.g. vision), to dynamically configure or ``tailor'' a manipulator so that it is matched to a particular situation and operation. This ``virtual tool'' is created by imposing customized, sensory modulated constraints between various degrees of freedom in the system. The remaining degrees of freedom constitute a small set of control parameters that are fitted to a particular operation. We argue that, within the confines of fairly broad application domains, a small set of ``tool classes'' can be defined that will serve as a general purpose sensory-motor toolbox for a wide variety of applications. We further argue that such class definitions can be made portable not only across tasks, but across platforms as well. The implementation of a number of basic tool classes, on various platforms, using vision and other sensory modalities, is described; and their use in performing multi-stage sensory-modulated manipulation tasks is illustrated.