Eye contact in video conferencing
Today’s technology seems to be able to reconstruct a 3D model almost in real-time from video input. Take a look here: Vi3Dim . Also, people can construct a 3D human face model in quite good quality now: Morphable Face: Automatic 3D Reconstruction and Manipulation from Single Data-Set Reference Face .
Then I wonder how much it would be difficult to construct a 3D model of a person’s face (actually not only face but whole upper body being captured in webcam) in real-time and show the rendered animation from this model instead of webcam video data. We could use two webcams to get more realistic and fast 3D model from stereographic analysis.
Why would we want 3D rendered animated video instead of real video captured? First, you can chat with the other person looking INTO his/her eyes. Wouldn’t it be unnatural and awkward when you can tell the artifact from other person’s face? Yes, it would be. But if we can devise a very good algorithm which can mix the real 2D video stream and model animation so seamlessly that we cannot tell the difference, I believe there is a benefit of this way of video chatting. It is an emotional part. You will agree on that it really is important to look into each other’s eyes when we talk.
Compared to the whole spectrum of the application fields of 3D reconstruction, making eye contact in video chatting may be just a minor, insignificant side job. I know. It’s all about how well we would be able to construct 3D models from 2D video data. It could be used as a person’s identity element in security check possibly in parallel with voice recognition. It could be used in a brand new way of user interface with eye tracking. Just imagine a cursor is following where you are looking at. Spooky? (♥Aug.2010 RiverWay)