To allow nonintrusive active visual model acquisition during a manipulation task, several requirements must be met:
To motivate the above requirements, consider that solving a typical manipulation task involves several manipulation models. In different phases of the manipulation with a robotic arm and hand system, different motor DOF's are used. In each phase a particular projection of motor signals to actuators is selected. For example coarse 3 DOF reaching moves are done using the long arm links while fine manipulation in 6 DOF can be done by either the fingers on the robot hand or by also using the wrist of the robot arm. The manipulation model in which we are interested is the one of the grasped object. The motor part of the model then depends on the projection function, which is known, but could be difficult to invert, the grasp point, and how the grasp points vary over time (slips etc). The last two are typically either unknown or only approximately known.
So while previous work (e.g. [Hollinghurst and Cipolla, 1993, Hager et al., 1993]) have successfully calibrated a visual-motor system using only a linear calibration, this has been done by estimating the linear transformation between a viewing geometry model (e.g. affine) and the Cartesian space of a calibrated robot arm. This works when the manipulated object is rigidly attached to the calibrated arm, but not for an uncalibrated arm or object-hand-arm system.
An estimate to the image Jacobian can be
obtained by physically executing a set of calibration movements
along the basis
directions of motor space ([Conkie and Chongstitvatana, 1990, Hollinghurst and Cipolla, 1993, Hager et al., 1995])
and approximating the Jacobian with finite differences:
where
gives the size of the calibration movements.
However to keep the estimate current for the typically nonlinearly
changing environment, the agent would need to perform these movements
rather frequently. This is not appropriate in most manipulation
settings, where the calibration movements would interfere with the
task.
Partial modeling of the viewing geometry using
an ARMAX model and estimating only one or a few parameters
(e.g. depth) has also been tried [Feddema and Lee, 1990, Papanikolopoulos and Khosla, 1993]. This however
restricts the camera-robot configurations and environments to
structured, easy to model settings.
We seek instead an online method, which estimates the Jacobian by simply
observing the process, without introducing any extra ``calibration''
movements. In observing the process we obtain the changes in
visual appearance
corresponding to a
particular controller command
. This is essentially
a secant approximation of the derivative of f along the direction
. We want to update the Jacobian in such a way so as
to satisfy our most recent observation (secant condition):
.
The condition above is under-determined; thus a family of updating formulas, called the Broyden hierarchy, is defined as follows:
(15)
where
are different rank 1 matrices so that the rank of the
correction term
is equal to the number of non-zero
.
We choose an unsymmetric correction term:
This is a first order updating formula in the above hierarchy, and it
estimates the Jacobian in all directions after n orthogonal moves
. It
trivially satisfies the secant condition, but to see how
it works let us consider a change of basis transformation P
to a coordinate system O' aligned with the last movement so
.
In this basis the correction term in eq. 4.8 is zero except
for the first column. Thus
our updating schema does the minimum change necessary
to fulfill the secant condition (minimum change criterion).
Let
be n orthogonal
moves, and
the transformation matrices as above.
Then
are just circular permutations of
, and
the updating schema is identical to the finite differences in eq.
4.6, in the
frame.
Note that the estimation in eq. 4.8
accepts movements along arbitrary directions
and thus needs no additional data other than what is available
as a part of the manipulation task we want to solve.
In the more general case of a set of non-orthogonal moves
, the Jacobian gets updated along the dimensions
spanned by
.
Over a course of time our estimation method will generate a
piecewise
linear estimate of the visual-motor model. This is illustrated in
fig.4.2.
Estimated visual-motor manipulation models are useful in a variety of settings. In the next chapter
we show how to use them for visual servo control without the need of prior models on both robot arms and hands. In Chapter 6 we show how to use the inverse visual-motor model to generate images, thus simulating the actions of an articulated active agent. Internally in the system the visual-motor models are useful in a variety of ways. They can serve as visual representations for recognition, and as models for filtering, tracking and search reduction. In most on-line applications (e.g. control, visual tracking prediction and on-line view synthesis) the current (time varying) Jacobian estimate of the visual motor model is sufficient. In off-line applications (e.g. off-line view synthesis of long movies, off-line motion planning) a more global estimate is needed, since no real-world feedback is available to update the local model. Our system acquires these global models incrementally simply by storing (presently without any forgetting) the past values of the time (state) varying local models. In off-line applications the visual-motor model is approximated by a spline function interpolating the sparse stored model (we use first or third order spline bases).