3d

To be precise, we're talking about breakthrough technology from the Henson Digital Puppetry Studio, which is part of the Jim Henson Company's Creature Shop. Some of the company's early technology received an Academy Award for Technical Achievement, but the work that led to their recent Engineering Emmy is even more impressive: motion capture with real-time rendering and 3D compositing, in full quality, full resolution.

Read that last sentence slowly. Real time. 3D Models. 3D compositing. Ready for broadcast and streaming output.

No rendering.

The technologies are simple enough by themselves: "mechanical hand controls, a control computer, and a digital puppet workstation which renders the live on-screen image of the character." It also integrates technologies from AMD and NVIDIA to display finished quality 3D rendering from mental ray, and composited into a virtual environment in real time.

If you're interested in reading more, the Henson Company's page has a lot of wonderful information. But first, let's roll tape. This is a clip from "Sid The Science Kid," airing on PBS Kids. As you watch it, keep in mind that all the performances are taking place and being fully rendered and composited in real time.





Did I mention real time?

The fact is that motion capture for films and games has been around for years. In fact, MOST games have some kind of motion capture. The character of Gollum in The Lord of the Rings was entirely a motion capture performance. Henson is one of the companies doing this kind of thing. Those are most definitely NOT real time processes.

The animation on Sid the Science Kid is in its own way quite primitive -- but not the motion. It's flexible, fluid, and dynamic...because that's how people are. The camera swoops through space...because that's how cameras are. The action is taking place in a studio with people, and being output in full quality, full resolution, composited in virtual 3D spaces, in real time.

Here's a nifty peek behind the scenes from the Wall St. Journal.





Variety adds some technical detail: "For any given project, as many as six such characters can interact at once, their every move tracked by 36 infrared cameras and played back in real-time on one of six huge screens surrounding the stage." The article also points out that for shows like Sid, the real-time output goes into Maya, where a Mental Ray pass adds nuances like fluid dynamics for the hair.

In the end, producing a fully-rendered 3D show takes about as much time to produce and post as a regular 2D sitcom. And even without a finishing pass, the output is ready for broadcast and streaming. As Henson begins licensing this technology, and hardware and software continues to refine, expect much bigger things to happen very, very quickly.


++++

Henson has of course been doing much more than Muppets over the past 50+ years. Yes, Henson's puppetry goes back to the mid-1950s, although for most of us, it begins with Sesame St. in 1969. I'll save for another time the stories of how Sesame St. was my primary influence for creating nature documentaries (no kidding), but it's enough for now to observe that you've seen Henson creatures in The Dark Crystal, Labyrinth, Farscape, and many, many others.


(Although Yoda was voiced by Frank Oz, who also voiced Cookie Monster, Bert, Miss Piggy, Henson and Co. had nothing to do with the design or performance of Yoda.)


The Henson Creature Shop recently put together a reel that captures some of the wide range of styles they've worked in over the years.




++++

Last but not least, I've always been a huge fan of the music woven into Henson Company productions, going back to the songs from Sesame Street, and the great Muppet Movie songs (including the Academy Award -nominated "Rainbow Connection." Later, after you've finished reading this, check out Willie Nelson's leisurely swinging, sweetly heartfelt rendition. Def Jam has disabled embedding, but follow the link to YouTube.



.


But first, here's the one I can't get out of my head, from our boy Sid the Science Kid. Schoolhouse Rock for 21st century indie kids: "I Love Charts." Seriously, one of the best new songs I've heard this year...maybe even the last couple of years. I think you're going to dig it as much as I do.

I noted in my entry on the Tron sequel that a speedy look at Joseph Kosinski’s IMDb profile reveals virtually nothing -- not even his birthday. I’m amazed that nobody has gotten around to it yet, but he doesn’t even have an entry at Wikipedia.

(I wonder if it's related to the absence of an entry on the Tron sequel. Probably. The only power in the universe I can think of that's more powerful than the masses converging on Wikipedia is Disney.)

And so we ask,





who is this man, and why is he directing the sequel to Tron?

I first discovered him a year before any mention of him helming the Tron sequel, the same way that millions of other folks did: I saw this AMAZING commercial for the Xbox 360 videogame, “Gears of War.” On top of footage of horrific battles and a massive, terrifying monster, he lays a haunted, heartbreaking version of “Mad World.” It makes for an unsettling mix of violence, sorrow, humility, fear, and overwhelming mortality.

Hit the HQ button. Set it to full screen. Turn it up.





Here’s Kosinski talking about his work on the spot, which, remarkably enough, started with the song.





That version of “Mad World” is by Gary Jules, and was first heard by most of us in the remarkable “Donnie Darko.”





"Donnie Darko" was enough to propel the song to #3 in the UK in 2003, but its presence in the Gears of War commercial drove it to #1 at iTunes in 2006.


If you like that, you should also check out the full-length version. It lacks the blunt-force trauma of the shorter version, but it gives you a stronger sense of Kosinki’s cinematic vision. Again, click the HQ button. Watch full screen. Turn it up.






What I did IMMEDIATELY after seeing that first clip was to find out as much as I could about it. I quickly found the director’s website, josephkosinski.com. It turns out that he’s directed quite a few very, very high-impact spots over the years – even if, to be honest, I still haven’t seen most of them outside his website.

The website is in Flash, so links beyond that one are a no-go. The site is also a little old, I think – the clips are small-ish, and dog slow. It’s still worth poking around – lots of great info about the spots, including credits...but watch the spots here before you visit.

One of his commercials is among my recent favorites: “Lincoln Effect,” and it includes the great tagline, “Starships Don’t Need Keys.”





Since we’re talking about directing “Tron 2.0,” I want to draw your attention to “Apple, iSPEC,” a short film that, according to the credits, “postulates the evolution of the personal media device and experience, placing the viewer within a digital recreation of the Colorado Lounge from Stanley Kubrick’s ‘The Shining.’”

Sound freaky? Well, it is. It also evokes a postulatory (I guess) evolution of the world of Tron, from one oriented around the dark, to one oriented around light. Regardless, the camera moves through the opening scenes, including a very interesting new software UI, exactly as you might imagine it moving through a Tron sequel.





That one was all CG, and frankly looks it. Not in a bad way, but for all that it offers a strongly personal camera perspective, there are clearly no people in this environment. It’s truly gorgeous, though, and I’m not shocked that it won the Autodesk iDesign Award.

I learned that from a brief bio formerly posted at the site for his (former?) company KDLAB. The site’s just a landing page now, pointing you elsewhere, but I found the bio floating around, apparently untethered.

Also jumping out at me from his bio: graduating from “Stanford University with a BS in Mechanical Engineering in 1996 and from Columbia University with a Masters of Architecture in 1999. Since then, he has taught Advanced Digital Design at Columbia and serves on the beta board for Discreet in the development of their next-generation design software.”

Let’s add this up: engineering, plus advanced design, plus next-gen software development, equals TRON, baby!

To really, really see this pay off, check out “Nike, LesJumelles.” That’s French for “The Twins.” Watch it first, and then we’ll talk.





A profile at Autodesk’s website talks with Kosinski about using an alpha version of 3D Studio Max and some other Autodesk tools to put it together.

He offers a slightly more artistic take at Archinect, where he discusses the "twin" motif. Here’s the money quote:

“I did some investigation into the relationship between speed and energy and rediscovered Einstein's concept of “Time Dilation” - something which had always seemed fascinating to me. Basically, it states that as you approach the speed of light, time seems to slow down for you, and speed up for everything else that isn't moving.”

Man, oh, man, I cannot WAIT for this movie! After reading that, and seeing those, even if I’d never heard Tron, I’d want to see a full-length version of whatever this cat is up to.

That first Gears of War I saw came soon after “LesJumelles,” when Kosinski had moved to Venice, CA’s "Anonymous Content"...after being recruited by one David Leo Fincher, who is credited as “Creative Consultant” on "Gears of War, Mad World.”





A last note about college degrees in mechanical engineering and architecture, and work in the world of software design: it’s not all that many steps from building devices, to building buildings, to building a world. Because whatever else is true about the world inside the game of Tron, it reflects the strong mechanical and engineering design of a software/hardware mind. It’s not enough for the world inside Tron to be beautiful. It has to visually make SENSE.

I’m going to end this post where it began, with the original version of “Gears of War, Mad World.” This clip opens with that, followed by “making of” footage from Digital Domain, with block renderings and motion capture footage intercut with the final version. Like many of the best magic tricks, it becomes even more impressive once you see how it’s done. You’ll see the technology, but you’ll also see how Kosinski and his team turn it into art.





Kosinski is our boy, all right.


Bonus clips:

It turns out that our boy is quite at home with sequels. I showed his sequel to Gears of War, above. Here’s the third sequel. You know the drill: HQ. Full screen. Turn it up.





And here’s his sequel to “Les Jumelles.”





Here's a 2005 montage of his work. Most of my favorite bits are in the “Les Jumelles” and “iSPEC” pieces, but check the intro: very, VERY Tron.





And a special bonus digression on commercial directors who, like Fincher, transitioned to features. Here are a few off the top of my head:

Errol Morris (The Fog of War), Jonathan Glazer (Sexy Beast), Spike Jonze (Being John Malkovich), Tony Scott (Top Gun, and more others than you remember), Mark Romanek (24 Hour Photo), Tony Kaye (American History X), Michel Gondry (Eternal Sunshine of The Spotless Mind), Antoine Fuqua (Training Day), and Ridley Scott (take your pick).

(Who am I leaving out? Let me know in the comments.)

I should also note that some of these guys have done some of the best music videos of all time. Romanek: “Constant Craving,” "Are You Gonna Go My Way," “Closer,” and one of the all-time greats, Johnny Cash’s version of “Hurt." (I wrote about it here.)






Fincher’s videos are so off the hook that, as with Kosinski, I looked him up the first time he caught my, with Madonna’s “Express Yourself,” another on my short list for best ever.






Check it full screen, and loud. Some obvious nods to “Metropolis,” only with crotch grabbing. “Rated M, for Mature” – no kidding -- but a real joy to watch again. Throw in “Vogue,” “Forever Your Girl,” “Janie’s Got a Gun,” "End of the Innocence," and “Cradle of Love" off the top of my head.

Anyway, I like commercials. I like music videos. I like movies. Storytelling is storytelling.

Which brings us back to Kosinski and the Tron sequel. I have no idea if the guy liked the first Tron, or even if he saw it at the time. (I'm guessing he was around 8 when it came out.) But as I look at his work again, I can’t imagine anybody better equipped to direct the second one.

Jeffry Katzenberg has been a major advocate of stereoscopic 3D for a while. The first place I saw him talk about it was ShoEast in 2007, a gathering of eastern US movie theater owners and operators. The talk was called "The Future of 3D in the Digital Age," and it was among the places that he said that all of Dreamworks' animated features would be released as stereoscopic.

On one count, he underestimated how quickly this would come to pass: he said that there would be "5, 6 or 7 'Super-A' titles" in 2009, 12 to 18 by 2010. So far this year, we've already seen My Bloody Valentine, the Jonas Brothers concert movie, Henry Selick's "Coraline," and just this weekend, "Aliens vs. Monsters," with 10 more slated for release this summer.

Here's the first trailer for Aliens vs. Monsters, which first ran in 3D during the Super Bowl:




Here's the second trailer:





Here's the third. Not sure why YouTube won't let me embed the high-quality version, but it's worth following the link to take a gander.


The next biggie is going to be Ice Age: Dawn of the Dinosaurs.






(You can see Entertainment Weekly's preview of the year's 3D releases here, but it's worth trying to track down the print issue, which includes anagplypic stills from most of those features, along with 3D glasses to view 'em. Pretty slick. It also doesn't have much to say about James Cameron's Avatar, opening in December...but I sure will later.)

So for the pace of "Super 'A'" releases, we're well ahead of schedule.

As theater owners pushed back on Katzenberg in 2007, they said that upgrading screens was going to be way, way too expensive. His reply was that they'd make more money on 3D showings, and this week's premiere of "Monsters vs. Aliens" certainly bore that out. Here's how the numbers broke down:

• $58.2 million for the weekend, on 4104 screens
• 28% of those screens were 3D, and accounted for 58% of the gross!
• .03% --that's three-tenths of 1 percent -- of those 4104 screens were IMAX (143 to be exact)...and they accounted for nearly TEN PERCENT of the gross! (Again to be exact, $5.2 million, a record.)

To put it another way, 28.3% of the screens accounted for nearly 70% of the money that Av.M pulled in! That is, as the kids say, off the hizz-OOOK! (Actually, I doubt any kids say that anymore.) There's absolutely no question that Katzenberg is being proved right on feature after feature.

However, here's one area he was a little optimistic on: he predicted 6000 3D-equipped screens by March 2009. The number is more like 2000 in the US, with a smattering more overseas. Let's be generous and call it halfway there.

The good news is that there are a total of 8000 screens "committed" (see previous link)...but as far as I can tell, no particular timetable.

When I say 8000 screens, the vast majority are those offered by RealD. Dolby is late to the party, but has a few hundred screens committed, mostly in Europe. The draw is that they don't require a new screen, and sell their gear outright. RealD requires new screens and collects royalties....but c'mon, seriously now, RealD has it locked up. Although as far as overall revenues, all signs point to Katzenberg being exactly right: the real money is in 3D exhibition.

It's true now. It's going to get truer. Expect more details.

In the meantime, our stereoscopic 3D coverage is going to be a wild ride that you're not going to want to miss. If you don't already subscribe, now's a good time to start.