In an active or behavioral vision system the acquisition of visual information is not an independent open loop process, but instead depends on the active agent's interaction with the world. Also the information need not represent an abstract, high level model of the world, but instead is highly task specific, and represented in a form which facilitates particular operations. Visual servoing, when supplemented with on-line visual model estimation, fits into the active vision paradigm. Results with visual servoing and varying degrees of model adaption have been presented for robot arms [1, 3, 2, 5, 6, 9, 13, 15, 16, 18]. Visual models suitable for specifying visual alignments have also been studied [19, 8, 7]. However, the focus of this work has been the movement (servoing) of the robot, not on on-line estimation of high DOF visual-motor models. In this paper we focus on the model exploration aspect and present an active vision technique, having interacting action (control), visual sensing and model building modules, which allows the simultaneous visual-motor model estimation and control (visual servoing) of a variety of robotic active agents.
We place the following requirements on the active visual model acquisition:
A combined model acquisition and control approach has many advantages. In addition to permitting uncalibrated visual servo control, the on-line estimated models are useful for (1) prediction and constraining search in visual tracking [13, 2], (2) perfomring local coordinate transformations between manipulator (joint), world, and visual frames[9, 13], and (3) synthesizing views from a basis of agent poses . We have found such an adaptive approach to be particularly helpful in robot arm manipulation when carrying out difficult tasks, such as manipulation of flexible material [9, 13], or performing large rotations for exploring object shape . For a dextrous multifinger robot hand, such as the Utah/MIT hand, the fully adaptive approach is appealing because dextrous manipulation of a grasped object is much harder to model accurately than a typical robot arm system, where the object is rigidly attached to the end effector.
In this paper we present four main contributions: