Archive for January, 2010

ALL THESE WORLDS ARE YOURS: Leslie Ekker on VFX for ‘2010′ – the Movie, Not the Year

2010_21_630x354
Well, it’s finally 2010. As you know, Pan Am currently offers commercial flights to all the major space stations; every family has pet dolphins in their specially-converted cetacean-friendly homes; computer graphics have finally hit 16-bits, displayed on futuristic CRT monitors; and the United States and the Soviet Union are on the brink of war.

Okay, so maybe the film 2010, Peter Hyams’ 1984 sequel to Stanley Kubrick’s 2001: A Space Odyssey, got a few details wrong. And it’s not really on the same level as its classic predecessor. But it’s still a fun, smart, great-looking sci-fi adventure that deserves a second look.

Roger Ebert said it better:

Once we’ve drawn our lines, once we’ve made it absolutely clear that 2001 continues to stand absolutely alone as one of the greatest movies ever made, once we have freed 2010 of the comparisons with Kubrick’s masterpiece, what we are left with is a good-looking, sharp-edged, entertaining, exciting space opera…

Just as the year 1984 spurred interest in the novel 1984, so 2010 has created renewed interest in the film – Google searches for “2010 movie” have spiked sharply in the last two months, and the film is up 413% in popularity this week on IMDb.

To satisfy those succumbing to the current 2010 mania, I spoke to Zoic Studios commercial creative director Leslie Ekker, who was a member of the miniatures crew for the film.

2010_16_630x354

“The first thing we had to do on 2010 was to build the spaceship Discovery from 2001. Unfortunately, in England, where the ship was built and shot and stored, an accountant had decided years before not to pay for the storage of the ship anymore; drew a line through a number on a list; and all the models were destroyed. There was literally nothing surviving. But we had to reproduce the ship as exactly as possible so that people would recognize it. And the only way we could do it – none of the drawings existed, no information, no photographs—was to rent a laserdisc of the film; freeze-frame it; take photographs of those frames; enlarge them to the point where they were useful for me; and do overlays, tracing the edges of all the details onto a drawing. Then I did a perspective analysis, and created six orthographic views that could be used as construction drawings. I had to do that with the entire Discovery, front-to-back, in order to be able to reproduce it.

“The production was in touch with the original people. In fact, all the visual effects were being produced by Doug Trumbull, who was one the principle people on the team for 2001. He knew all the people involved, and got in touch with the right folks — but nobody had anything left. Pretty sad, considering what a classic 2001 was.

2010_18_630x354

“So first I had to do these construction drawings, and it was challenging, because the shots [in the original 2001] are actually fairly scarce. There aren’t a lot of things from different angles, and of course the image quality was pretty poor. So there was a lot of interpretation. Ultimately, we got it pretty close.

“We made two different scale models of the Discovery, and one large-scale model of the front end of the ship. One model was about 10 feet long, much smaller than the original ones they built in England. They built huge miniatures due to the shorter depth of field of lenses in those days! Ours was designed to rotate, as well. In the scenes where they come upon the Discovery still orbiting, it’s tumbling end-over-end because of precession, the physical force on a rotating body (its gravity carousel) that is 90 degrees to any other forced applied to it.

“The Discovery is dusted down with sulfur, because it’s orbiting around [Jupiter’s moon] Io, which has sulfur volcanoes that erupt into space. So that got stuck to the body of Discovery, it’s all sulfur yellow — so naturally our models were painted yellow, unlike the original.

“The Boss Film model shop supervisor was Mark Stetson, an Oscar-winning feature film VFX supervisor now. In his model shop in Marina Del Rey, we built a lot of different miniatures for the movie. Some were of the Leonov, the Russian ship, and the Discovery; but also of the moons’ surfaces. We built a few models that were pretty interesting.

2010_36_630x354

“One of the ideas they explored in 2010, that actually had a lot of controversy surrounding it, was the concept of life under the ice on Jupiter’s moon Europa. They have since found there is most likely liquid water under that ice, and it possibly could have enough warmth to support life; and it may actually harbor life, maybe in bacterial form. It’s hard to say. That was kind of interesting. One of my jobs on the movie was to help make that life.

“We built the surface of Europa, a small section of it, and filled it with some water, sections of ice, and strange looking plants. We used Madagascar palms for some of the plants, because they’re so strange looking already; they look quite alien. In the shallow water of the pond, built into the tabletop of the model, we had some invisible rigging that could move some very fine feathery plants in an intelligent way, as if they were motivated, under the surface of the water. That’s what you see in the film when you see something moving under the water — it’s actually a very fine dried plant getting pulled around by an invisible rig.”

2010_8_630x354

The design of the Russian spaceship, the Leonov, had to differ from the “American” design of the Discovery. “The common wisdom was that Russian technology looks heavier, and feels clunkier, and has more exposed detail, kind of a brutal design style. [Legendary industrial designer] Syd Mead was employed to design the Leonov, and did some beautiful drawings.

“Peter Hyams, the director of the film, scrutinized the drawings very closely to make sure every single line from the drawing was on our model; to the point where, in a perspective construction drawing, if a sketched line ran off the corner of an object, he wanted a little wire glued onto the object to represent that line. It was kind of strange, but we did it.

“I spent about six weeks just building plumbing in the hub of the rotating section. If you look carefully at the Leonov, there’s this really intricate rat’s nest of pipes of all different sizes, weaving in and out and going off in different directions. And there was one on each side, so they had to match. I had to make matching sets of this very intricate piping, melting and bending pieces of plastic model piping by hand. It took weeks and weeks to do. Then I had to make a miniature version, half that size, for the smaller scale Leonov. It was a lot of fun, but it was also really challenging.

“One of the other things I did was to create the Cyrillic typeface you see on the side of the Leonov, and the other graphics that go on the ship. We had a translator create all the different words we needed, and then went to a type house and had wax transfers made — these were rub-downs we used to use in the graphic days before computers. I had sets and sets of them made in both the different scales, applied them to the ship, and then we painted them into the overall paint scheme of the ship. It’s the only time I’ve had to work in Russian!

2010_11_630x354

“There’s a sequence in the film where the Leonov has to execute an aerobraking maneuver. That’s when a spacecraft just grazes the outer atmosphere of a planet, using aerodynamic friction to slow itself down, rather than burning fuel. It does this with a device called a ballute, which is a half-balloon half-parachute. We were had to make ballutes that were deployed from the core of the aft-end of the Leonov, and they were big inflatable airbags — gas bags, really. I had to develop a way to create airtight bags that were of a very specific shape. The surface pattern on them looked like some kind of fiber-reinforced textile. We had to be able to stow them in a very small volume, from which they would inflate very quickly to a certain size on camera. And then we made a separate set of those same ballutes that were fully inflated to a rigid shape.

“We also needed to make another set of ballutes, coated with pyrotechnic powder, and light them on fire, send them down a wire and film them, to be composited with the rest of the spacecraft for the actual moments of high friction and heat. So it was quite a project, and I was assigned the task of designing and producing these things.

“I had to learn pretty fast how to make airtight structural bags out of very tough, heat-resistant materials. I used very thin Mylar, like space-blanket material; and thin double stick tape to make the seams. I made screen prints of the graphic pattern on the surface. And we ended up using a leaf blower to inflate them. Leaf blowers are great, because they pump huge volumes of air at low pressure. You can inflate something very large without a lot of force behind it, so when it reaches the end of its inflation capacity it doesn’t burst a seam. After about five weeks of effort, that actually worked.

“Then we set about sculpting the rigid versions, which were just foam sculptures that were hard-coated, and painted and stenciled with the same graphic pattern as the airbags. Then we made copies in fire-resistant epoxies, in order to pyro-coat them and do the actual burning sequences. All this work was done at Boss Films’ Glencoe model-making facility, where there’s nothing but condos now. In those days Glencoe was all shipyards and industrial facilities; that’s all gone now.”

2010_19_630x354

Ekker remembers 2010 as a fun, if challenging, experience. He also related an anecdote on how his work on the film helped him in another way:

“When you’re in the union, you have a card in a file that tells what your specialties are. And in the union system, if a model shop is putting together a union crew, they have to just call the union and say ‘send me five model makers,’ and hope they get good people. A lot of people, who say they’re model makers, really are not model makers.

“The workaround was, you would go and request someone who had a skill that was very specific to that person. A lot of us had skills that were very unique-sounding, but they were legitimate, because we had to be able to do the skill. After 2010, my skill card said, “pneumatic inflatable structures,” and “foreign language typesetting for model making” — skills so esoteric, it could only be me. So if, say, someone wanted to hire me, they could call up the union hall, and say “I need a guy who can make an airbag,” and they’d send me up!

For more info: 2010 on Wikipedia, IMDb, Amazon; Roger Ebert’s review.

, , , , , , , , , , , , , , , , , ,

2 Comments

Zoic Studios’ ZEUS: A VFX Pipeline for the 21st Century

zeus_v_greenscreen_630x354Actors Christopher Shyer and Morena Baccarin on the greenscreen set of ABC’s V; the virtual set is overlaid.

Visual effects professionals refer to the chain of processes and technologies used to produce an effects shot as a “pipeline,” a term borrowed both from traditional manufacturing and from computer architecture.

In the past year, Zoic Studios has developed a unique pipeline product called ZEUS. The showiest of ZEUS’ capabilities is to allow filmmakers on a greenscreen set to view the real-time rendered virtual set during shooting; but ZEUS does far more than that.

Zoic Studios pipeline supervisor Mike Romey explains that the pipeline that would become ZEUS was originally developed for the ABC science fiction series V. “We realized working on the pilot that we needed to create a huge number of virtual sets. [Read this for a discussion of the program's VFX and its virtual sets.] That led us to try to find different components we could assemble and bind together, that could give us a pipeline that would let us successfully manage the volume of virtual set work we were doing for V. And, while ZEUS is a pipeline that was built to support virtual sets for V, it also fulfills the needs of our studio at large, for every aspect of production.

“One of its components is the Lightcraft virtual set tracking system, which itself is a pipeline of different components. These include InterSense motion tracking, incorporating various specialized NVIDIA graphics cards for I/O out, as well as custom inertial sensors for rotary data for the camera.

“Out of the box, we liked the Lightcraft product the most. We proceeded to build a pipeline around it that could support it.

“Our studio uses a program called Shotgun, a general-purpose database system geared for project shot management, and we were able to tailor it to support the virtual set tracking technology. By coming up with custom tools, we were able to take the on-set data, use Shotgun as a means to manage it, then lean on Shotgun to retrieve the data for custom tools throughout our pipeline. When an artist needed to set up or lay out a scene, we built tools to query Shotgun for the current plate, the current composite that was done on set, the current asset, and the current tracking data; and align them all to the timecode based on editorial selects. Shotgun was where the data was all stored, but we used Autodesk Maya as the conduit for the 3D data – we were then able to make custom tools that transport all the layout scenes from Maya to The Foundry’s Nuke compositing software.”

By offloading a lot of the 3D production onto 2D, we were able to cut the cost-per-shot.

Romey explains the rationale behind creating 3D scenes in Nuke. “When when you look at these episodic shows, there’s a large volume of shots that are close-up, and a smaller percentage of establishing shots; so we could use Nuke’s compositing application to actually do our 3D rendering. In Maya we would be rendering a traditional raytrace pipeline; but for Nuke we could render a scanline pipeline, which didn’t have same overhead. Also, this would give the compositing team immediate access to the tools they need to composite the shot faster, and it let them be responsible for a lot of the close up shots. Then our 3D team would be responsible for the establishing shots, which we knew didn’t have the quality constraints necessary for a scanline render.

“By offloading a lot of the 3D production onto 2D, we were able to cut the cost-per-shot, because we didn’t have to provide the 3D support necessary. That’s how the ZEUS pipeline evolved, with that premise – how do we meet our client’s costs and exceed their visual expectations, without breaking the bank? Throughout the ZEUS pipeline, with everything that we did, we tried to find methodologies that would shave off time, increase quality, and return a better product to the client.

“One of the avenues we R&Ded to cut costs was the I/O time. We found that we were doing many shots that required multiple plates. A new component we looked at was a product that had just been released, called Ki Pro from AJA.

“When I heard about this product, I immediately contacted AJA and explained our pipeline. We have a lot of on-set data – we the have tracking data being acquired, the greenscreen, a composite, and the potential for the key being acquired. The problem is when we went back to production, the I/O time associated with managing all the different plates became astronomical.

“Instead of running a Panasonic D5 deck to record the footage, we could use the Ki Pro, which is essentially a tapeless deck, on-set to record directly to Apple ProRes codecs. The units were cost effective – they were about $4,000 per unit – so we could set up multiple units on stage, and trigger them to record, sync and build plates that all were the exact same length, which directly corresponded to our tracking data.”

We found methodologies that would shave off time, increase quality, and return a better product to the client.

Previously, the timecode would be lost when Editorial made their selects, and would have to be reestablished. “That became a very problematic process, which would take human intervention to do — there was a lot of possibility for human error. By introducing multiple Ki Pros into the pipeline, we could record each plate, and take that back home, make sure the layout was working, and then wait for the editorial select.” The timecode from the set was preserved.

“The ZEUS pipeline is really about a relationship of image sequence to timecode. Any time that relationship is broken, or becomes more convoluted or complicated to reestablish, it introduces more human error. By relieving the process of human error, we’re able to control our costs. We can offer this pipeline to clients who need the Apple ProRes 442 codec, and at the end of the day we can take the line item of I/O time and costs, and dramatically reduce it.”

Another important component is Python, the general-purpose high-level programming language. “Our pipeline is growing faster than we can train people to use it. The reason we were able to build the ZEUS pipeline the way we have, and build it out within a month’s time, is because we opted to use tools like Python. It has given us the ability to quickly and iteratively develop tools that respond proactively to production.

“One case in point – when we first started working with the tracking data for V, we quickly realized it didn’t meet our needs. We were using open source formats such as COLLADA, which are XML scene files that stored the timecode. We needed custom tools to trim, refine and ingest the COLLADA data into our Shotgun database, into the Maya cameras, into the Nuke preferences and Nuke scenes. Python gave us the ability to do that. It’s the glue that binds our studio.

“While most components in our pipeline are interchangeable, I would argue that Python is the one component that is irreplaceable. The ability to iteratively making changes on the fly during an episode could not have been deployed and developed using other tools. It would not have been as successful, and I think it would have taken a larger development team. We don’t have a year to do production, like Avatar – we have weeks. And we don’t have a team of developers, we have one or two.

While most components in our pipeline are interchangeable, Python is the one component that is irreplaceable.

“We’re kind of new to the pipeline game. We’ve only been doing a large amount of pipeline development for two years. What we’ve done is taken some rigid steps, to carve out our pipeline such a way that when we build a tool, it can be shared across the studio.”

Romey expects great things from ZEUS in the future. “We’re currently working on an entire episodic season using ZEUS. We’re working out the kinks. From time to time there are little issues and hiccups, but that’s traditional for developing and growing a pipeline. What we’ve found is that our studio is tackling more advanced technical topics – we’re doing things like motion capture and HDR on-set tracking. We’re making sure that we have a consistent and precise road map of how everything applies in our pipeline.

“With ZEUS, we’ve come up with new ways that motion capture pipelines can work. In the future we’d like to be able to provide our clients with a way not only to be on set and see what the virtual set looks like, while the director is working — but what if the director could be on set with the virtual set, with the actor in the motion capture suit, and see the actual CG character, all in context, in real-time, on stage? Multiple characters! What if we had background characters that were all creatures, and foreground characters that were people, interacting? Quite honestly, given the technology of Lightcraft and our ability to do strong depth-of-field, we could do CG characters close-to-final on stage. I think that’s where we’d like the ZEUS pipeline to go in the future.

“Similar pipelines have been done for other productions. But in my experience, a lot of times they are one-off pipelines. ZEUS is not a pipeline just for one show; it’s a pipeline for our studio.

“It’s cost effective, and we think can get the price point to meet the needs of all our clients, including clients with smaller budgets, like webisodes. The idea of doing an Avatar-like production for a webisode is a stretch; but if we build our pipeline in such a way that we can support it, we can find new clients, and provide them with a better product.

“Our main goal with ZEUS was to find ways to make that kind of pipeline economical, to make it grow and mature. We’ve treated every single component in the pipeline as a dependency that can be interchanged if it doesn’t meet our needs, and we’re willing to do so until we get the results that we need.”

For more info: Lightcraft Technology; InterSense Inc.; Shotgun Software; AJA Video Systems; IDYE’s coverage of V.

, , , , , , , , , , , , , , , , , , , ,

5 Comments

The End of Rendering: Zoic Studios’ Aaron Sternlicht on Realtime Engines in VFX Production

Zoic created this Killzone 2 commercial spot entirely within the Killzone 2 engine.

The level of the technology available to produce computer graphics is approaching a new horizon, and video games are part of the equation.

Creators in 3D animation and visual effects are used to lengthy, hardware-intensive render times for the highest quality product. But increasingly, productions are turning to realtime rendering engines, inspired by the video games industry, to aid in on-set production and to create previz animations. Soon, even the final product will be rendered in realtime.

Aaron Sternlicht, Zoic Studios’ Executive Producer of Games, has been producing video game trailers, commercials, and cinematics since the turn of the millennium. He has charted the growth of realtime engines in 3D animation production, and is now part of Zoic’s effort to incorporate realtime into television VFX production, using the studio’s new ZEUS pipeline (read about ZEUS here).

Sternlicht explains how realtime engines are currently used at Zoic, and discusses the future of the technology.

“The majority of what we do for in-engine realtime rendering is for in-game cinematics and commercials. We can take a large amount of the heavy-lifting in CG production, and put it into a game engine. It allows for quick prototyping, and allows us to make rapid changes on-the-fly. We found that changing cameras, scenes, set-ups, even lighting can be a fraction of the workload that it is in traditional CG.

“Right now, you do give up some levels of quality, but when you’re doing something that’s stylized, cel-shaded, cartoonish, or that doesn’t need to be on a photo-realistic level, it’s a great tool and a cost effective one.

We’re going to be able to radically alter the cost structures of producing CG.

“Where we’re heading though, from a production standpoint, is being able to create a seamless production workflow, where you build the virtual set ahead of time; go to your greenscreen and motion capture shoot; and have realtime rendering of your characters, with lighting, within the virtual environment, shot by a professional DP, right there on-set. You can then send shots straight from the set to Editorial, and figure out exactly what you need to focus on for additional production — which can create incredible efficiencies.

“In relation to ZEUS, right now with [ABC’s sci-fi series] V, we’re able to composite greenscreen actors in realtime onto CG back plates that are coming straight out of the camera source. We’re getting all the camera and tracking data and compositing real-time, right there. Now if you combine that with CG characters that can be realtime, in-engine rendered, you then can have live action actors on greenscreen and CG characters fully lit, interacting and rendered all in realtime.

“People have been talking about realtime VFX for the last 15 years, but now it’s something you’re seeing actually happening. With V we have a really good opportunity. We’re providing realtime solutions in ways that haven’t been done before.

“Now there’s been a threshold to producing full CG episodic television. There has been a lot of interest in finding a solution to generate stylized and high quality CG that can be produced inexpensively, or at least efficiently. A process that allows someone to kick out 22 minutes of scripted full CG footage within a few weeks of production is very difficult to do right now, within budgetary realities.

“But with in-engine realtime productions, we can get a majority of our footage while we’re actually shooting the performance capture. This is where it gets really exciting, opening an entire new production workflow, and where I see the future of full CG productions.”

What game-based engines have Zoic used for realtime rendering?

“We’ve done a several productions using the Unreal 3 engine. We’ve done productions with the Killzone 2 engine as well. We’re testing out different proprietary systems, including StudioGPU’s MachStudio Pro, which is being created specifically with this type of work in mind.

“If you’re doing a car spot, you can come in here and say ‘okay, I want to see the new Dodge driving through the salt flats.’ We get your car model, transfer that to an engine, in an environment that’s lit and realtime rendered, within a day. We even hand you a camera, that a professional DP can actually shoot with on-site here, and you can produce final-quality footage within a couple of days. It’s pretty cool.”

How has the rise of realtime engines in professional production been influenced by the rise of amateur Machinima?

“I’ve been doing game trailers since 2000. I’ve been working with studios to design toolsets for in-game capture since then as well. What happened was, you had a mixture of the very apt and adept gamers who could go in and break code, or would use say the Unreal 2 engine, to create their own content. Very cool, very exciting.

“Concurrently, you had companies like Electronic Arts, and Epic, and other game studios and publishers increasing the value of their product by creating tool sets to let you capture and produce quality game play — marketing cameras that are spline-based, where you can adjust lighting and cameras on-the-fly. This provided a foundation of toolsets and production flow that has evolved into today’s in-engine solutions.”

It’s truly remarkable how the quality level is going up in realtime engines, and where it’s going to be in the future.

How has this affected traditional producers of high-end software?

“It hasn’t really yet. There’s still a gap in quality. We can’t get the quality of a mental ray or RenderMan render out of a game engine right now.

“But the process is not just about realtime rendering, but also realtime workflow. For example, if we’re doing an Unreal 3 production, we may not be rendering in realtime. We’ll be using the engine to render, instead of 30 or 60 frames a second, we may render one frame every 25 seconds, because we’re using all the CPU power to render out that high-quality image. That said, the workflow is fully realtime, where we’re able to adjust lighting, shading, camera animation, tessellation, displacement maps — all realtime, in-engine, even though the final product may be rendering out at a non-realtime rate.

“Some of these engines, like Studio GPU, are rendering out passes. We actually get a frame-buffered pass system out of an engine, so we can do secondary composites.

“With the rise of GPU technology, it’s truly remarkable how the quality level is going up in realtime engines, and where it’s going to be in the future. Artists, rather than waiting on renders to figure out how their dynamic lighting is working, or how their subsurface scattering is working, will dial that in, in realtime, make adjustments, and never actually have to render to review. It’s really remarkable.”

So how many years until the new kids in VFX production don’t even know what “render time” means?

“I think we’re talking about the next five years. Obviously there will be issues of how far we can push this and push that; and we’re always going to come up with something that will add one more layer to the complexity of any given scene. That said, yes, we’re going to be able to radically alter the cost structures of producing CG, and very much allow it to be a much more artist-driven. I think in the next five years… It’s all going to change.”

Read Zoic Studios’ ZEUS: A VFX Pipeline for the 21st Century.

, , , , , , , , , , , , , , , , , , , , , ,

4 Comments