MEETMIKE: Pushing the Limits of Live, Real-Time Animation
MEETMIKE is the name given to a VR experience demonstrated at SIGGRAPH 2017 conference – the same one where we had the chance to see the fascinating Neurable.
The joint effort of Mike Seymour (the “puppeteer”, thus the name MEETMIKE), Cubic Motion (face image tracking & solving), 3Lateral (facial rig), Wikihuman (Mike Seymour scan) and a slew of other companies has resulted in a significant overall improvement of real-time rendering technology.
End result? MEETMIKE, looking as real as it gets, rendered in real-time at 90 frames per second in VR using Unreal Engine.
If you haven’t seen MEETMIKE yet, do it now! Here’s a video:
Factsheet:
- MEETMIKE contains about 440,000 triangles being rendered in real time, which means rendering of VR stereo about every 9 milliseconds. Hair consumes a whopping 75% of all triangles.
- There are 80 joints in Mike’s face rig. Again, majority of those are used for hair and facial hair movement.
- Face mesh requires only about 10 joints for jaw, eyes and the tongue.
- The final version of the head mesh combines these joints with around 750 blendshapes
- The system relies on complex traditional software design and as much as three deep learning AI engines.
It’s almost unbelievable, right?!
Now, what if I showed you a real-time rendering presentation that won the award for the best real-time graphics & interactivity last year at SIGGRAPH 2016?
Hellblade: Senua’s Sacrifice
Looks quite impressive, right?
What you just watched, all of that live performance capture and real-time rendering, is the backbone for Hellblade: Senua’s Sacrifice.
Hellblade: Senua’s Sacrifice is a single-player adventure set in the Viking age, developed by an indie studio Ninja Theory. It follows a young female Celtic warrior Senua, trapped in the personal hell of psychosis and schizophrenia. Throughout the game, Senua is traversing the pits of hell in search of her deceased lover’s soul, but in reality, she’s battling her mental illness.
It has just been launched on Steam (August 8), and it has received overwhelmingly positive reviews so far.
Now, the game itself is far from VR ready. But, all the facial shots of Senua, as well as her whole body, share much of the same technology required to run MEETMIKE.
Without going too deep into the matter, it’s obvious that live, real-time rendering technology is within our grasp. It’s slowly finding its way into the mainstream gaming and video animation.
Amazing future awaits us!
Unfortunately, most of the current hardware and tech is at its limits when it comes to live, real-time rendering and performance capturing. Add VR into the mix and you are really pushing it.
For those reasons, we’re still a few years away from performance capture rendering becoming an industry staple.
However, progress is inevitable, and with new generations of graphics cards, real-time rendering will surely become a viable option for developers and studios.
Imagine – a VR game where you meet somebody you know in real life. The person’s real-time rendered in-game avatar looks exactly the same as the person does in real life.
Fascinating. And scary.
