Over the last few years, depth cameras have become increasingly popular for a range of applications, including human-computer interaction and gaming, augmented reality, machine vision, and medical imaging. Many of the commercially available devices use the time-of-flight principle, where active illumination is temporally coded and analyzed on the camera to estimate a per-pixel depth map of the scene.

This demonstration proposes a fundamentally new imaging modality for all time-of-flight (ToF) cameras: per-pixel velocity measurement. The proposed technique exploits the Doppler effect of objects in motion, which shifts the temporal frequency of the illumination before it reaches the camera. Using carefully coded illumination and modulation frequencies of the ToF camera, object velocities directly map to measured pixel intensities. A slight modification of the imaging system allows for color, depth, and velocity information to be captured simultaneously. Combining the optical flow computed on the RGB frames with the measured metric axial velocity allows further estimation of the full 3D metric velocity field of the scene. This technique has applications in many computer graphics and vision problems, for example motion tracking, segmentation, recognition, and motion deblurring.

Since the ToF camera system captures video at real-time frame rates, many exciting applications can be demonstrated. Attendees can swing a baseball bat, punch in the direction of the camera, or throw a baseball in the direction of the camera. The system reconstructs the full-velocity image from the Doppler shift and displays the measured speed and reconstruction.