This week Snapchat updated its app to include a layer of different “lenses” for taking a selfie. Upon clicking your face, a spider web detects your image within a second, and then a slew of silly, scary, and plain weird digital sketches decorates your tongue, teeth, and eyes in real time. It even makes sounds if you move in a certain way.
This is what Dr. Hao Li would call realtime facial animation with on-the-fly correctives. And that spider web is what he would call an “efficient mesh deformation algorithm” and it can track any individual, starting from just a single face scan of the subject in a neutral pose.
Li did something similar at the Santa Monica Rotary meeting Friday, Sept. 12. He turned his entire face into a monkey. And then a panda.
Li was demonstrating the future of three-dimensional computer graphics and shared his vision for making human 3D visualization standard in every home.
“Recently, we have figured out how to display high quality displays at low cost. So these things can be democratized,” Li said.
Li has been described as the rising star of the USC Viterbi computer graphics department, where he is a tenure-track assistant professor in the Geometric Capture Lab. The department is involved in everything from facial geometry, to geometry processing, urban city modeling, and computer animation and physically based simulation.
His algorithms on dynamic shape reconstruction and non-rigid registration are widely used in the industry, ranging from leading visual effects studios to defense projects and manufacturers of state-of-the-art radiation therapy systems.
Li has consulted for everyone from Facebook to Disney and has film credits from “Star Treck Into Darkness,” “The Hobbit: The Battle of the Five Armies,” and “Captain America: The Winter Soldier,” to name a few. He’s also received the Google Faculty Research Award and the Okawa Foundation Research Grant in 2015 and was named one of the world’s top 35 innovator under 35 by MIT Technology review in 2013 and NextGen 10: Innovators under 40 by C-Suite Quaterly in 2014.
Of Taiwanese descent, raised in Germany, and taught at a French school, Li’s background is as unconventional as his hairstyles. Though his work speaks for itself, Li has a quirky, almost punk-y, side to him that makes his resume all the more fascinating. He cut out the jargon and got down to the basic fundaments of 3D imaging, in his professorial manner, asking Socratic questions and making sure everyone followed along.
One could imagine how much his students want to come to class and work with him. He’s young, he’s bright, and he has fun with what he does.
One of the most interesting snippets of his talk was when it came to Hollywood. Deconstructing the science of special effects, Li revealed how much time and work actually goes into a CGI-dominated film. For example, creature animation in Hollywood uses ” sophisticated suits that have sensor balls attached to the actor’s body, as seen in something like “Planet of the Apes.” With a camera in front of the actor’s face, designers then re-animate another creature onto the face.
Li called the “The Curious Case of Benjamin Button” one of the best examples of a highly realistic CGI face, which was how lead actor Brad Pitt appeared throughout the whole movie.
“It took three weeks to do 10 seconds of animation using an entire studio,” Li said.
That timeline hasn’t changed since 2008, as Li said it took the same amount of time when he worked on the 3D facial tracking for animating the digital Paul Walker in “Furious 7.”
“But this isn’t something an ordinary person can use,” Li said, moving away from special effects in the film industry and focusing more on gaming and other uses for 3D technology.
To bring it into the home, you have to rely on hardware, digital devices, to digitize the person or thing, Li said. The vision is to bring this technology of 3D printing and 3D imaging into virtual reality.
“In order to improve 3D content, we’re creating technology that would allow you to digitize things from the physical world to the virtual work. How does that work? Using the technology of a 3D scanner,” Li said, either through lasers or a camera.
Some of these devices are all ready available in the home, like Microsoft’s Xbox One Kinect, which can be used with a 3D builder app to scan and 3D print in full-color.
“It can also be used to scan entire three-dimensional human bodies,” Li said.
3D scanners can help in the medical replique montre industry, where some hospitals have all ready begun using noninvasive scanning for tumors. You can also create visual models for fashion or fitness, “where you can basically stand in front of the camera and basically analyze your body without having to be measured.”
In other words, the computer can tell if you’ve been putting on weight. Other uses are on the horizon. 3D scanning and printing are the future, and Li wants to see it as soon as possible.
“If we scan a person, we can extract everything about the person.”
For more information about Santa Monica Rotary, visit rotaryclubofsantamonica.org.
