An Auto-Multiscopic Projector Array for Interactive Digital Humans

Emerging Technologies

 

  View Video

An Auto-Multiscopic Projector Array for Interactive Digital Humans

Automultiscopic 3D displays allow a large number of viewers to experience 3D content simultaneously without the hassle of special glasses or head gear. This display uses a dense array of 216 video projectors to generate images with high angular density over a wide field of view. As users move around the display, their eyes smoothly transition from one view to the next. The display is ideal for displaying life-size human subjects, as it allows for natural personal interactions with 3D cues such as eye-gaze and spatial hand gestures.

The installation presents "time-offset" interactions with recorded 3D human subjects. A large set of video statements was recorded for each subject, and users access these statements through natural conversation that mimics face-to-face interaction. Conversational reactions to user questions are retrieved through speech recognition and a statistical classifier that finds the best video response for a given question. Recordings of answers, listening, and idle behaviors are linked together to create a persistent visual image of the person throughout the interaction. This type of time-offset interaction can support a wide range of applications, from creating entertaining performances to recording historical figures for education.

Andrew Jones
University of Southern California

Jonas Unger
Linköpings universitet

Koki Nagano
University of Southern California

Jay Busch
University of Southern California

Xueming Yu
University of Southern California

Hsuan-Yueh Peng
University of Southern California

Oleg Alexander
University of Southern California

Paul Debevec
University of Southern California