An active vision agent has control over its actions, and can observe the results of an action via the changes in visual appearance. We study a robot agent, fig. 1, in an unstructured environment. The robot action reference frame is joint space, described as desired joint angles, , and their time derivatives . The changes in visual appearance are recorded in a perception or feature vector . Visual features can be drawn from a large class of visual measurements[1, 9], but we have found that the ones which can be represented as points positions or point vectors in camera space are suitable. We track features such as boundary discontinuities (lines,corners) and surface markings. Redundant visual perceptions ( ) are desirable as they are used to constrain the raw visual sensory information.
Figure 1: Visual control setup using two cameras.
The visual features and the agent's actions are related by a visual motor transfer function f, satisfying . The goal for the control problem is, given current state and , the desired final state in visual space , to find a motor command, or sequence thereof, s.t. . Alternatively, one can view this problem as the minimization of the functional .
In a traditional, calibrated setting we have to know two functions, camera calibration h and robot kinematics g, either a priori, or through some calibration process. The accuracy at which we can represent objects and have the active agent manipulate them depends on the accuracy of both of these functions ( ), since typically feedback is only over the agents internal joint values.
In our uncalibrated visual servoing the visual-motor transfer function is unknown and at any time k we estimate a first order model of it, . The model is valid around the current system configuration , and described by the ``image'' or visual-motor Jacobian defined as
The image Jacobian not only relates visual changes to motor changes, as is exploited in visual feedback control but also highly constrains the possible visual changes to the subspace of (Remember ). Thus the Jacobian J is also a visual model, parameterized in exactly the degrees of freedom our system can change in and useful in a variety of active vision tasks as we will explore in this paper. During the execution of a manipulation task the agent successively learns more and more about the environment in the form of a piecewise linear model on an adaptive size mesh.