To be precise, we're talking about breakthrough technology from the Henson Digital Puppetry Studio, which is part of the Jim Henson Company's Creature Shop. Some of the company's early technology received an Academy Award for Technical Achievement, but the work that led to their recent Engineering Emmy is even more impressive: motion capture with real-time rendering and 3D compositing, in full quality, full resolution.
Read that last sentence slowly. Real time. 3D Models. 3D compositing. Ready for broadcast and streaming output.
No rendering.
The technologies are simple enough by themselves: "mechanical hand controls, a control computer, and a digital puppet workstation which renders the live on-screen image of the character." It also integrates technologies from AMD and NVIDIA to display finished quality 3D rendering from mental ray, and composited into a virtual environment in real time.
If you're interested in reading more, the Henson Company's page has a lot of wonderful information. But first, let's roll tape. This is a clip from "Sid The Science Kid," airing on PBS Kids. As you watch it, keep in mind that all the performances are taking place and being fully rendered and composited in real time.
Did I mention real time?
The fact is that motion capture for films and games has been around for years. In fact, MOST games have some kind of motion capture. The character of Gollum in The Lord of the Rings was entirely a motion capture performance. Henson is one of the companies doing this kind of thing. Those are most definitely NOT real time processes.
The animation on Sid the Science Kid is in its own way quite primitive -- but not the motion. It's flexible, fluid, and dynamic...because that's how people are. The camera swoops through space...because that's how cameras are. The action is taking place in a studio with people, and being output in full quality, full resolution, composited in virtual 3D spaces, in real time.
Here's a nifty peek behind the scenes from the Wall St. Journal.
Variety adds some technical detail: "For any given project, as many as six such characters can interact at once, their every move tracked by 36 infrared cameras and played back in real-time on one of six huge screens surrounding the stage." The article also points out that for shows like Sid, the real-time output goes into Maya, where a Mental Ray pass adds nuances like fluid dynamics for the hair.
In the end, producing a fully-rendered 3D show takes about as much time to produce and post as a regular 2D sitcom. And even without a finishing pass, the output is ready for broadcast and streaming. As Henson begins licensing this technology, and hardware and software continues to refine, expect much bigger things to happen very, very quickly.
++++
Henson has of course been doing much more than Muppets over the past 50+ years. Yes, Henson's puppetry goes back to the mid-1950s, although for most of us, it begins with Sesame St. in 1969. I'll save for another time the stories of how Sesame St. was my primary influence for creating nature documentaries (no kidding), but it's enough for now to observe that you've seen Henson creatures in The Dark Crystal, Labyrinth, Farscape, and many, many others.
(Although Yoda was voiced by Frank Oz, who also voiced Cookie Monster, Bert, Miss Piggy, Henson and Co. had nothing to do with the design or performance of Yoda.)
The Henson Creature Shop recently put together a reel that captures some of the wide range of styles they've worked in over the years.
++++
Last but not least, I've always been a huge fan of the music woven into Henson Company productions, going back to the songs from Sesame Street, and the great Muppet Movie songs (including the Academy Award -nominated "Rainbow Connection." Later, after you've finished reading this, check out Willie Nelson's leisurely swinging, sweetly heartfelt rendition. Def Jam has disabled embedding, but follow the link to YouTube.
.
But first, here's the one I can't get out of my head, from our boy Sid the Science Kid. Schoolhouse Rock for 21st century indie kids: "I Love Charts." Seriously, one of the best new songs I've heard this year...maybe even the last couple of years. I think you're going to dig it as much as I do.
Posted by: Tim Wilson on Aug 30, 2009 at 10:33:28 am
