next up previous
Next: 3 Visual Measures Up: Estimation of Visual-Motor Models Previous: 1 Introduction

2 Viewing model

  An active vision agent has control over its actions, and can observe the results of an action via the changes in visual appearance. We study a robot agent, fig. 1, in an unstructured environment. The robot action reference frame is joint space, described as desired joint anglesgif, tex2html_wrap_inline734 , and their time derivatives tex2html_wrap_inline736 . The changes in visual appearance are recorded in a perception or feature vector tex2html_wrap_inline738 . Visual features can be drawn from a large class of visual measurements[1, 9], but we have found that the ones which can be represented as points positions or point vectors in camera space are suitable[10]. We track features such as boundary discontinuities (lines,corners) and surface markings. Redundant visual perceptions ( tex2html_wrap_inline740 ) are desirable as they are used to constrain the raw visual sensory information.

   figure49
Figure 1: Visual control setup using two cameras.

The visual features and the agent's actions are related by a visual motor transfer function f, satisfying tex2html_wrap_inline744 . The goal for the control problem is, given current state tex2html_wrap_inline746 and tex2html_wrap_inline748 , the desired final state in visual space tex2html_wrap_inline750 , to find a motor command, or sequence thereof, tex2html_wrap_inline752 s.t. tex2html_wrap_inline754 . Alternatively, one can view this problem as the minimization of the functional tex2html_wrap_inline756 .

In a traditional, calibrated setting we have to know two functions, camera calibration h and robot kinematics g, either a priori, or through some calibration process. The accuracy at which we can represent objects and have the active agent manipulate them depends on the accuracy of both of these functions ( tex2html_wrap_inline762 ), since typically feedback is only over the agents internal joint values.

In our uncalibrated visual servoing the visual-motor transfer function is unknown and at any time k we estimate a first order model of it, tex2html_wrap_inline766 . The model is valid around the current system configuration tex2html_wrap_inline768 , and described by the ``image''[15] or visual-motor Jacobian defined as

(1) displaymath779

The image Jacobian not only relates visual changes to motor changes, as is exploited in visual feedback control but also highly constrains the possible visual changes to the subspace tex2html_wrap_inline770 of tex2html_wrap_inline772 (Remember tex2html_wrap_inline740 ). Thus the Jacobian J is also a visual model, parameterized in exactly the degrees of freedom our system can change in and useful in a variety of active vision tasks as we will explore in this paper. During the execution of a manipulation task the agent successively learns more and more about the environment in the form of a piecewise linear model on an adaptive size mesh.


next up previous
Next: 3 Visual Measures Up: Estimation of Visual-Motor Models Previous: 1 Introduction

jag@cs.rochester.edu