next up previous contents
Next: 9.2 External Prosthesis Up: 9 Future Work Previous: 9 Future Work

9.1 Remote Tele-Assistance

The robotic hardware for the remote tele-assistance would ideally be a mobile base able to carry the weight of a manipulator consisting of a robot arm and wrist, and a multi-fingered gripper (robot hand). The vision hardware would be a stereo ``eye-in-hand'' camera pair mounted on the robot arm, with independent vergence, but other eye movements effected by the robot arm. A visual front end capable of feature and motion tracking and a graphics processor with two frame rate video channels for overlays would also be included. The user interface can be either a head mounted stereo display (VR helmet) or a stereo 3D VDU for display, and a choice of a 3D pointing device (3D mouse), or a gaze position eye-tracker for the input device.

The remote human operator views the environment around the mobile manipulator through the stereo cameras and VR helmet. In viewing mode, the robot arm will change the viewpoint in response to the human head movements. Using either the 3D mouse to click on objects, or fixating on them, the user can select objects to be manipulated. By ``clicking'' on a menu of manipulation ``virtual tools'', much like in a Mac Paint type drawing program, the operator selects the kind of manipulation he or she wants to do, and to instantiate the manipulation the operator specifies the desired visual alignments in the scene, again using either the mouse or gaze direction.

In this application several of the methods described in this thesis are useful. The visual motor model estimation and control we described in Chapter 5 is used to bring about movement. The high level visual space task specification and planning described in Chapter 7 is used to implement the man-machine interface, but with the more advanced display (VR helmet or 3D display) and pointing devices (3D pointer, or eye-tracker) for input and output, instead of the simple workstation VDU and computer mouse. In cases where system delays are long, the visual simulation method described in Chapter 6 is used to augment the real visual feedback with synthesized data showing the manipulation movements based on their predicted appearance before the real visual feedback arrives. (System delays may be long due to distance as in remote manipulation in space, due to limitations in the transfer network as in manipulation over the Internet, or just due to slow response of the robotic system.)


next up previous contents
Next: 9.2 External Prosthesis Up: 9 Future Work Previous: 9 Future Work

Martin Jägersand