The structure of the visual planning and control system is shown in Fig. 7.3. Open loop manipulations described in any of the three frames, namely visual, (local) world or joint, are stored in the canned motor actions module. These are typically used to bring about some immediate goal, such as insertion, screwing in, dropping off etc. Since no model updating is done, these actions can be used only for short motions near the point of the last estimated model.
In the high level trajectory planner, task level information is
used to break a single task goal into several subgoals. For instance
to insert a puzzle piece, as is shown in Fig. 7.6, the overall
goal is broken into several subgoals; namely, bring the manipulator
above the puzzle piece (visual goal
just above the piece),
pick up the puzzle piece (goal
in contact with the piece),
move up above the piece (goal
),
move the piece above the hole (goal
) etc. The low level
trajectory generator takes this coarse sequence of goals and for
convergence and error control reasons (see Section 5.2
and [Jägersand and Nelson, 1994]) breaks
it down further along straight lines in
connecting the
subgoals
.
Figure 7.3: Overall structure of our system, and the time
frames for running each part.
Real time visual feature trackers of three different kinds are used to obtain visual information. The Oxford snakes [Curwen and Blake, 1992] are used to track surface discontinuities. A locally developed parallel convolution tracker tracks multiple arbitrary surface markings and corners, and for reliability in repeated experiments, or to deal with smooth featureless surfaces, such as the light bulb in Fig. 7.7, we use special purpose trackers, tracking glued-on tracking targets or small lights. To improve tracking, viewing geometry models are widely used. For instance, the Oxford snake package uses an affine model to constrain the motion of the spline control points to rigid 3 D deformations, and a strain energy model for non-rigid image plane deformations. The convolution trackers use point velocity and acceleration for prediction.