2008-171 – High-Presence, Low-Bandwith, Apparent 3D Video-Conferencing with a Single Camera

Photo by Jackson SmithJeffrey R. Spies with (onscreen, left to right) Timothy R. Brick and Steven M. Boker, Ph.D. Executive SummaryUniversity of Virginia researchers have developed a novel method for enhancing users92 video-conferencing experience by enabling apparent three-dimensionality with the use of a single camera. High-presence and direct eye-contact conversations are achieved under low-bandwidth requirements without the need for expensive motion-capture technologies or multi-camera arrays.BackgroundVideo-conferencing has become an increasingly popular way to engage in personal and business-related communications, with noted drawbacks stemming from participants92 lack of physical co-presence and technical demands for high bandwidth. Typically, due to the physical displacement between a user92s camera and screen, participants cannot simultaneously watch and make eye contact with other users on-screen. This physical displacement alters micro-facial expressions that would normally occur in face-to-face meetings and may make participants feel uncomfortable, potentially damaging conversations and relationships. While current technologies exist to enhance co-presence, they often consist of elaborate motion-tracking systems and multi-camera arrays, which are expensive and demand high bandwidth requirements, making them undesirable in many video-conferencing situations.Invention DescriptionDeveloped by quantitative psychology researchers, the invention is a method for providing high-presence, real-time video-conferencing through simulated three-dimensionality. Using a single camera, each participant92s face and movements are captured, tracked and reduced to a few data points using statistical modeling. In this way, important facial features are captured and transmitted over very low bandwidth (a few hundred bytes per frame). At the remote device, motion parallax is used to reconstruct a simulated 3-D image that is modified to reflect the viewing angle and location of the recipient viewer, creating a near-photorealistic video of each participant92s face. This feature also allows apparent eye contact to be made and broken, as it is in in-person conversation.View a DemoView a brief demo movie of the technology in action.AdvantagesThe invention offers several advantages over conventional technologies:High-presence video-conferencing is achieved through a single camera, minimizing costs and the need for any special equipment Direct eye contact allows for more life-like conversation Use of statistical facial modeling allows for high-quality video transmission over low bandwidths, making it ideally suited for mobile technology Potential ApplicationsThe invention can be used to enhance a wide range of applications for which effective communication is imperative, such as:Video communications: enhancing conversations while reducing bandwidth, especially attractive for mobile devices Online gaming and animation: allowing users to manipulate avatar facial expressions Motion-picture industry: allowing easy post-alterations to an actor92s facial expressions, better reflecting a dubbed recording Medical, marketing and sports industries: increasing the consumer92s sense of physical presence at an event Marc Oettinger marc.oettinger@virginia.edu 434-982-1608