Framestore recently created roughly 550 shots for Guy Ritchie’s Sherlock Holmes: A Game of Shadows, including the film’s climactic encounter at the Reichenbach Falls and a thrilling battle on a speeding train, as well as numerous elaborate environmental shots and set extension work, a scene taking place inside a gun, and some ingenious, near-subliminal “puffs of breath.”
Shooting took place from September 2010 to January 2011, with the majority of first-unit material captured on location around the U.K. and at Elstree and Leavesden Studios. Supervising the shoots for Framestore were visual effects supervisor Sirio Quintavalle, compositing supervisor Kyle McCulloch and CG supervisor Ben White.
At Shadows’s gripping climax, Holmes and Moriarty meet in a fortress high in the Swiss Alps, built above the source of the colossal Reichenbach Falls. They play chess and then – as so often happens – turn to fisticuffs. Holmes realizes that the only way to definitively best his nemesis is to seize and pull him over the balcony into the waterfall below, even at the price of near-certain death. In a classic piece of Ritchie theater, the camera is placed midway down the falls, descending and pointing upwards, as the two figures locked together plummet towards it. They are moving faster than the camera, though, and it pans as they pass it. We switch to ultra-slow motion as we watch Holmes and Moriarty’s heads in close-up, inches apart and with very different expressions, before the film speed picks up again and we watch them plunging and separating as they continue their fatal descent. It is 29 seconds of virtuoso filmmaking, the sort that Ritchie has made his own. But the shot’s visual audacity placed high demands on the visual effects team.
VFX supervisor Chas Jarrett’s initial hope was that the sequence could be constructed purely through the use of real waterfall elements. Both Quintavalle and McCulloch traveled far and wide as part of the quest for the perfect falls. They both dangled perilously out of helicopters (Quintavalle in Switzerland, McCulloch in Norway), clutching their trusty Canon EOS 5D cameras as they captured plates and footage. In the end, no real waterfall proved up to the sequence’s demands. For starters, it would have had to be over a kilometer tall, a waterfall so high it was only possible to create it in the Soho offices of Framestore.
The actors – Robert Downey Jr. and Jared Harris – were filmed for the sequence using a technique that is becoming increasingly popular with VFX teams. They were harnessed to two gigantic KUKA robot arms. More commonly used for making cars, these powerful arms can be programmed to move – or puppeteer – the actors through space. While offering great benefits in terms of precision positioning and performance capture from the actors, this technique does have its downside. “There was an enormous quantity of clean-up work to be done before the actors were ready to be placed in the shots,” said McCulloch, “We had an eight-strong paint and roto team putting in long hours for many weeks to remove all traces of the harnesses and the arms, as well as patching up various bits of the outfits with CG elements.” This action was shot on a Phantom HD Gold camera, which can go up to 1000 fps, though 432 fps proved the ideal speed for the central slo-mo section.
The water itself was the biggest challenge. Ben White was Framestore’s CG supervisor for the project. “Several factors made this water more difficult than usual,” he said, “Waterfalls do not lend themselves to fluid simulation software, which is at its best on rivers and seas – things with a surface. Also, the fact that it was such a long waterfall meant that we were dealing with colossal amounts of data, because we had to have pre-roll for the entire kilometer, and when you factor in the time-warp down to almost 500 fps in the middle of the shot, and the fact that the camera is pointing straight up into the descending water, making it hard to see – all this meant we had our work cut out. In the end we broke up the water into several segments rather than trying to do it in one piece, with each segment being made of multiple layers to achieve the complexity we needed. We had a team working in Naiad, on the water coming down from above, and we had another team working in Houdini providing a different element for the lower part of the waterfall where it blends in to the live-action elements. We also added CG and live-action elements of water vapor and mist being pulled away from the surface which helped to give a special relationship to the falling actors in this huge space. The live-action elements were most successfully used when we’re looking straight down at the end, because essentially the camera’s locked off, so the photographed element worked well there.”
Another water element proved more readily susceptible to technological augmentation. As the protagonists pass the camera in close-up, Jarrett and Ritchie wanted to see water droplets in motion around their heads. The compositing team found that the latest version of Nuke (released just in time to help out) gave them a set of tools that helped them sidestep a 3D solution. “We found some photos of live action water droplets from reference libraries,” recalled Quintavalle, “And we put them onto cards – lots of droplets arranged in motion and distributed as we wanted. It was a temporary measure at first, while we waited for the ‘proper’ particles to get rendered. Then as it went along we thought, ‘Could we make them move a bit as well, add a bit of warping? Could we have a bit of refraction as you’re looking through them? Can we get the time-warp working properly?’ and each time the answer was yes. The drops just kept getting better within Nuke until we realized that we actually had the shot we needed.”
Half of Framestore’s 30-strong Shadows compositing team worked on this one sequence. It was one of the first to be started in October 2010, and among the last to be finished.
The other key sequence that Framestore created for Shadows takes place on a train speeding through nighttime countryside. A band of assassins, disguised as British soldiers, has boarded the train and plan on dispatching Holmes and Watson in a tornado of gunfire.
Except for one studio-shot carriage, Framestore created the entire train and the environments through which it passes. For interior work, there was a lot of greenscreen insertion of backgrounds as well as complementary reflections in the windows. As the fight develops, Holmes and Watson climb out onto the side of the train, so extensive digital environments were required. Many background plates were shot on panorama-cam and put in by compositing, but the entire foreground of the shot had sometimes to be digital, so White’s team created plants, bushes and scrub.
Other work by the CG team featured in the sequence includes a characteristically Ritchie journey through the inside of a gun which is pointing at Holmes as it is fired. We see the pin striking the percussion cap of the bullet and we then see the bullet fly past the camera ‘inside’ the barrel as it flies towards the end (which Holmes has sensibly spiked and which then blows up in the face of his would be assassin). “We pre-vized and conceived the shot working with Chas,” says White, “There is a lot of very subtle work in it – lots of attention to detail in terms of shading and texturing, as well as some lovely depth-of-field work put on by the compositing team.” White is also particularly pleased with the steam his team created for the train. “Steam is quite a challenge if you’re going to get the authentic roiling, dynamic, very voluminous looking stuff. We had a couple of guys working together – simulating in Houdini and then rendering it in fRibGen, our in-house rendering interface, and using a lot of proper volumetric shading written by the shading department. It was also an ideal chance to use FMote – a proprietary node-based particle processing tool which allows FX artists to manipulate particle data prior to rendering.”
The sequence featured dozens of digital set extension shots, which were accomplished by the 2.5D team using a combination of techniques, sometimes starting off in the CG department with some rough lighting passes, and then going to the digital matte painting team for their magic, and sometimes without CG input, using a combination of in-house tools in Maya and Nuke. A relatively straight-forward touch added by the comp team was the illusion of motion as the train rattles along. Choreographed to reflect the intensity of the action inside, the post-added motion impressed Ritchie enormously with the veracity it brought to the sequence.
Another subtle touch that pleased Ritchie and producer Joel Silver hugely was an effect that is all too easy to get wrong – the addition of tiny puffs of condensation caused by characters as they speak and breathe in a cold environment. The Swiss Alps castle scenes were all shot in a studio, so to convey the atmosphere of a freezing Alpine night, the Framestore team came up with an ingenious solution. “We developed a reference system using real filmed footage from a cold room,” explained Quintavalle, “We found different types of ‘puff’ representing different syllables, and then used the audio soundtrack to trigger the appropriate puff. It’s a nice hybrid of real elements with an automated approach and was very controllable and quick. A couple of our artists turned out to be intuitively good at this, and they ended up doing 100 or so shots of breath enhancement. You’ll barely register it when you see it, but that’s really the point – it’s only good if you don’t notice it.”
Other important touches by the Framestore team included environmental recreations for establishing shots of Charing Cross station, Paris, and the environs around a gentleman’s club wherein Holmes faces another would-be assassin.
Sherlock Holmes: A Game of Shadows opened Dec. 16. The sequel to 2009’s box office hit, Sherlock Holmes, the film sees Robert Downey Jr. and Jude Law reprising their roles as Sherlock Holmes and Dr. Watson, with Guy Ritchie once more at the helm. Produced, like the first film, by Joel Silver, Lionel Wigram, Susan Downey and Dan Lin, Shadows also sees the return of cinematographer Philippe Rousselot and visual effects supervisor Chas Jarrett.