An active vision agent has control over its actions, and can
observe the results of an action via the changes in visual appearance.
We study a robot agent, fig. 1, in an
unstructured environment. The robot action reference frame is
joint space, described as desired joint angles
,
,
and their time derivatives
. The changes in
visual appearance are recorded in a perception or feature
vector
. Visual features can be drawn from
a large class of visual
measurements[1, 9], but we have found that the ones which
can be represented as points positions or point vectors in camera space are
suitable[10].
We track features such as boundary
discontinuities (lines,corners) and surface markings. Redundant visual
perceptions (
) are desirable as they are used to constrain
the raw visual sensory information.
Figure 1: Visual control setup using two cameras.
The visual features and the agent's actions are related by a visual motor
transfer function f, satisfying
.
The goal for the control problem is, given current state
and
, the desired final
state in visual space
, to find a motor command,
or sequence thereof,
s.t.
. Alternatively, one can view this problem as the minimization of
the functional
.
In a traditional, calibrated setting we have to know
two functions, camera calibration h and robot kinematics g,
either a priori, or through some calibration process.
The accuracy at which we can represent objects and have the active
agent manipulate them depends on the accuracy of both of these
functions (
), since typically feedback is only
over the agents internal joint values.
In our uncalibrated visual servoing the visual-motor transfer function
is unknown and at any time k we estimate a first order model of it,
.
The model is valid around the current system configuration
, and
described by the ``image''[15] or visual-motor Jacobian
defined as
(1)
The image Jacobian
not only relates visual changes to motor changes, as is
exploited in visual feedback control but also highly
constrains the possible visual changes to the subspace
of
(Remember
).
Thus the Jacobian
J is also a visual model, parameterized in exactly the degrees
of freedom our system can change in and useful in a variety of
active vision tasks as we will explore in this paper.
During the execution of a manipulation task the agent successively
learns more and more about the environment in the form of a piecewise
linear model on an adaptive size mesh.