Autonomous Driving

Photo-realistic simulation

Autonomous driving is a good example of an application where it is necessary to combine perception (vision) and control. However, testing such a system in the real world is difficult and potentially dangerous, especially in complex dynamic environments such as urban traffic.

Given recent advances in computer graphics, both in terms of the quality of the generated images and the rendering speed, we believe that a viable alternative to initial testing in the real world is provided by integrating photo-realistic simulation and real-time image processing. This allows testing the computer vision algorithms under a wide range of controllable conditions, some of which would be too dangerous to do in an an actual car.

Therefore, we built a simulator for driving in an urban environment that can be used both as a testbed for a driving program and with human subjects. A unique feature of this driving simulator is the ability to track eye movements within a freely moving VR helmet. This allows the assessment of exigencies in complex situations that can be used to guide the development of automated routines.

Driving simulator camcorder.gif (253 bytes)

Eye-tracking in virtual reality camcorder.gif (253 bytes)


Visual Routines for Autonomous Driving

Framework  camcorder.gif (253 bytes) Visual routines in simulated and real environments (4MB Mpeg)


Related publications


URCS Home Page