Tuesday, June 18, 2024
Subscribe Now

Voice Of The Crew - Since 2002

Los Angeles, California

HomeAwardsSpider-Man: No Way Home's Oscar-Nominated VFX Team on Divvying Up the Big...

Spider-Man: No Way Home‘s Oscar-Nominated VFX Team on Divvying Up the Big Action Scenes and Why Sandman Is Still Tricky


No Way Home
Image via Sony Pictures

With a worldwide gross of $1.8 billion and counting, Sony’s Spider-Man: No Way Home was not only the biggest movie of 2021, but it’s also now the third-highest-grossing movie of all time domestically, and that’s in spite of the ongoing global pandemic. The film owes much of its success to the way that Director Jon Watts and his team united three iterations of Spider-Man — Tom Holland, Andrew Garfield, and Tobey Maguire — plus a handful of past villains, including Alfred Molina‘s Doc Ock and Jamie Foxx‘s Electro. 

Of course, this superhero souffle required a ton of visual effects — reportedly 2,500 VFX shots — and the many technological advances that Hollywood has made since Sam Raimi‘s Spidey trilogy allowed the No Way Home team to improve upon the work done by their talented predecessors.

A few weeks back, Below the Line spoke with three of the visual effects experts responsible for pulling off this feat of movie magic, which includes pulling off the illusion of the entire film taking place in New York City despite actually filming in Georgia. 

Kelly Port served as No Way Home‘s overall VFX Supervisor after working on Marvel’s third and fourth Avengers movies and earning an Oscar nomination for his work on Infinity War. Chris Waegner is the Visual Effects Supervisor at Sony Pictures Imageworks and he has actually worked on five of the seven Spider-Man movies made by Sony. Scott Edelstein served as No Way Home’s VFX Supe at Digital Domain, and he also had a hand in Free Guy, which is also up for an Oscar this year.

At this point, if you’re reading this, you are likely to have already seen Spider-Man: No Way Home, but if you’re waiting for its VOD release later this month then be aware that a few plot points are discussed herein. So consider this your SPOILER WARNING!

Chris Waegner
Chris Waegner (via Imageworks)

Below the Line: Before we get into No Way Home, did any of you work on Spider-Man: Far from Home, or did you just come on for this one? I guess Chris worked on other Spider-Man movies, is that right?

Chris Waegner: Correct, on some of the other ones. I didn’t work on the last one. I was on a different movie at the time, but almost all of them during my tenure there.

Scott Edelstein: There are some other animators that worked on Spider-Man 2. Keith Smith was saying that he animated some of the Ock shots on Spider-Man 2, and then animated a bunch of them. He was a lead on this show, so that was cool. 

BTL: Chris, I know you’ve been at Imageworks for over 20 years and have touched upon almost all the other Spider-Man movies. No Way Home literally brings together characters from all of them, so did Imageworks still have some of the old assets that you were able to reuse, or did Jon want to start from scratch on a lot of it?

Chris Waegner: Working at Imageworks, yeah, we’ve worked on all of them, obviously, as have some other vendors. A lot of that stuff is online or on disk, or on tape, depending upon how it was brought online way back when. The technology has changed quite a bit. I think the other thing that played a role in the possibility of resurrecting some of these old model assets is, not only has the technology changed, but the characters’ physical proportions changed, as well. I mean, you’re looking at Tobey, who worked on the very early films. We have the model of Tobey. We used it as a visual proportional reference, but you know, he’s aged 15-plus years. Everybody’s physical shape has kind of changed, and that was pretty important to Jon and Kelly, to make sure that we were using these guys as digital doubles, so they have to match their new proportions.

Kelly Port
Kelly Port

BTL: Kelly, you’ve worked on the two most recent Avengers films, and those were obviously done by the time they had started on this one, so how did you get pulled over into the Spider-Man world after doing those movies?

Kelly Port: I think the relationship with Marvel was a big part of that, just having worked with them and worked with Disney before that. I’ve worked on Beauty and the Beast and Maleficent for the Disney side of things. I had known [Producer] Victoria Alonso, and a lot of the people at Digital Domain have sort of migrated over to Marvel, just like a weird proportion of them. I do have a lot of personal relationships at Marvel and have worked with them in the past, and obviously, having a good experience with Infinity War and Endgame. So I was tapped to do this one, and I was very honored. 

I was nervous. It was a big deal going into it, because we knew it was gonna be big. I don’t think anyone really knew it was gonna be this big, but that’s always a wonderful, pleasant surprise at the end. And it was really fun. It was a good experience. I was on it for about 25 months or so, and like all films, it gets a little into the grind towards the last few months, but I would say, all in all, it’s a very collaborative show with Chris and his team at Imageworks and with Scott and his team at Digital Domain, and all the other wonderful companies that worked on it. I think we had 12 in all.

BTL: At what point did you know about the villains that were being brought back? I feel like we were slowly getting things revealed with each trailer, but you also had villains like The Lizard and Sandman, which are mainly digital.

Port: Pretty early on, because I remember when I first met, especially with Chris and his team, and I think it was Luma, there was a handful of people that had worked on Sandman, and then mentioning Sandman, it’s like ‘Oh, boy,’ because that was tough. That was really, really tough back in the day. I think, technologically, that’s gotten a little bit… you guys can comment on this on your end. I don’t know if it’s gotten necessarily easier. I think you can iterate more, computing power and things like that have changed, and [the] software to create sand grain simulations. Those kinds of things, you’re able to iterate on them, and so you’re able to see them quicker, things like that. So that certainly helps, but still, that character is a very challenging character…

Scott Edelstein
Scott Edelstein

Edelstein: Like anything else, although all the technology has changed and gotten better and faster, everybody expects more and expects it to be cooler and better, and so it’s still hard.

Waegner: The visual demands are up there, and fortunately for us, for our sequence — the end battle — we were able to use Sandman slightly different. I remember telling the team to think of him as an earth elemental. He’s not made of sand — he’s made out of cinder blocks, concrete, any type of media that he could pull up from the island of the environment to make him this giant foe, if you will. We kind of used that to our advantage, so he’s like this tornado. Media’s coming into him. Physical media is kind of flying away from him. He’s just a big villainous storm, which was, I think, a lot of fun. We weren’t constrained to him being like a bipedal type of Sandman.

Edelstein: Everybody had a little different version of Sandman and Electro, too, right? Luma’s version, our version, your guys’ as well, Sandman was just a little bit different, right? Our Sandman’s on two legs, walking around an apartment building. Yours is this giant, villainous storm. And then it was Lumas that did the power corridor? That was more of like, he was that giant mound of sand for a bit — a little human in there, but kind of all over the place. It was cool, a lot of development on that one character.

Port: And we share those ideas, because like Scott was saying, the needs of the particular scene were different for these different sequences that Imageworks handled, versus Digital Domain or Luma, all handling Sandman and Electro but in different ways. He had slightly different costumes in some versus others, like Electro, for example. They all had sort of specific needs, but of course, we’re all sharing the stuff back and forth. It’s one of the challenges of sharing asset builds, where it’s not just an isolated scene. ‘Okay, this character only appears in this scene. You develop that character and go for it. We’ll do this with you.’ But since multiple characters  — well, almost really all of them — appeared in every single sequence, we had to very much collaborate and work off each other like, ‘Hey, okay, you’ll take the lead on this, but as we’re doing that, and they’re developing theirs, we’ll say, ‘Oh, this was developed. That’s really good. Can you incorporate some of that? That’s a cool look.’ Some of that feedback between companies.

No Way Home
image via Sony Pictures

BTL: I read that you had something like 2,500 VFX shots in the movie, so how did you go about dividing the movie up and deciding which vend0r was going to work on which section? Chris mentioned that they did the ending battle sequences, so how did you decide how to divide the film up?

Port: I think as you kind of break down the script, and we didn’t necessarily have a complete and full script from the time that these things needed to be done, but we had an idea of these big action beats, like, we knew that there’s going to be this bridge sequence where Ock was going to be revealed. We didn’t know exactly how big it was gonna be or things like that. We knew that there was going to be an end battle sequence, but we didn’t know where exactly it was going to be. There are still a lot of things that needed to be figured out along the way. This is a couple years back where this part was happening, but I think we had a pretty good idea. 

I know for Scott’s sequence and Digital Domain, they did the bridge sequence, but for the fight at Happy’s condo, that whole scene went through many, many different flavors, and a lot of it wasn’t going to end up at Happy’s condo initially at all. It was at totally different locations. There’s a lot of switching around and things like that that happened on that particular sequence. Same thing with Framestore and the Spidey-Strange chase. That didn’t really solidify until quite late in the process. We had all sorts of fun ideas, but sometimes you can get carried away with how much fun you’re having, and it’s like, ‘We probably should try to decide to do something pretty soon.’ [all three laugh at this]

Edelstein: When anything’s possible, and you have a bunch of creative people in a room, it can go on forever, right?

Port: Yeah, but we had a pretty good idea, and we broke down the script like that, and talked to each of the visual effects studios, like these guys are representing, and then Digital Domain did a couple big sequences, Imageworks did the huge sort of end battle sequence. Framestore did that Spidey-Strange chase, and then we did the powerline corridor and the opening sequence. You kind of divvy it up. A big thing about just Marvel, and I think a smart play in general, is that you don’t want to give too many shots if there’s a likelihood of things [that are] going to change dramatically, all the way up to the beginning. Because then, you just can’t handle that volume in such a quick turnaround. 

No Way Home
No Way Home Model Shot (image via Digital Domain/Sony)

BTL: Were you able to be on set, Kelly, or was there someone on set just to make sure what they were shooting would work with what was being developed at the VFX houses?

Port: I was in Atlanta. Part of this was pre-production, but we shot… let me think… October 2020 to March 2021 or something like that, in Atlanta. We were one of the first productions. I think Jurassic World: Dominion, they were the first to get into [putting everyone up in a hotel]. I think we were the second big one to start doing that, so a lot of stuff had to be figured out with COVID protocols and different color-coded groups, depending on how close you are to the actors and the directors and things like that. You had to be tested like three times a week for nine months — it was really fun.

Edelstein: They’re still testing you, just to make sure, right?

BTL: This entire movie takes place all in New York, unlike the last two movies. I’m not sure if you ever came to New York to shoot any of it, but what was involved with creating the movie’s New York setting without filming here?

Port: That was another thing that was highly impacted by COVID, because we wanted to try to shoot towards the fall, because that’s when it was mostly taking place, with the fall colors on the leaves and things like that. The crazy thing is that we just couldn’t because we were shooting towards the end of October and there [were] quarantine guidelines that we had to [follow], and there are risks of sending the director and actor there and then having them get back in time, because you’re quarantining, and actually get in time there to shoot again. So we ended up shooting a splinter crew in New York, just shooting backgrounds and reference. We did a few days of aerials. Just for the simplest things, like you would totally go there and shoot a guy like Flash, for example, standing on the corner, looking at his cell phone. We had to go there and shoot that background, measure the heck out of the camera, what the focal length was, its height and position, making sure that we can match that perspective later. And then shoot Flash on a blue screen later and try to match him in. Otherwise, that would have been an easy decision to just shoot in New York. You’d have a few weeks just in New York with the actors and the whole crew, but you had to do things really backward, just because we had to, given the circumstances.

Edelstein: New York has to be one of the most CG-built cities ever, right? How many times have all of our companies built and destroyed New York City?

Port: It’s surprising, even given that, because it’s so big, and any given shot, you want to make like a superhero shot, but that given shot, you see a few hero buildings and you see down a street maybe. Or maybe you see a wide one where you can get away with a little less projected detail. Yeah, it’s a lot of stuff, and a lot of companies have that. A few companies have gone into procedurally created buildings as well. That’s getting more sophisticated, certainly more than it was before. That’s a technique, but you also want to keep those iconic buildings too, especially like the Empire State Building, the Chrysler Building, all that. 

No Way Home
Animation shot (Digital Domain/Sony)

BTL: Chris, did you guys have any New York City from the Amazing Spider-Man that you were able to reuse, or would you combine those things with what was being done at other houses? 

Waegner: It’s kind of similar to like what Kelly was saying. We’ve built it historically, many different iterations, and then certain sequences tend to take place at certain areas of the city, so you kind of build out that kind of hero location. Through the years though, our New York City digital model has just gotten bigger and bigger and bigger. Now, we just added Liberty Island to it. I mean, before it was kind of something in the background out there. You see Liberty that they want to put it in a couple shots in the background, but it’s now just another thing that’s gonna get added to that giant library.

Port: And then, Scott’s section was not really in Manhattan, per se. What was that area?

Edelstein: Like, the Bronx-y area.

Port: Yeah, High Bridge, part of the Queens Expressway, basically. And then, the Happy’s condo stuff was right across from the UN, basically. 

Edelstein: Those are two areas that we’ve never built. We haven’t done any of those bridges, not to any kind of level that we had to build it for this. We had to build three square miles of high-res, photo-real New York City from that bridge that you could render in the daytime. [laughs]

BTL: VFX really missed an opportunity, because when the pandemic hit New York City between March and June, you could literally drive around these areas getting plate shots, because there was no one out there. You could have LIDAR scanned every single block of the city.

Edelstein: Everybody was talking about doing it.

Port: Well, we scanned every single block that we needed.

BTL: What’s one shot or sequence that each of you is the proudest of in terms of how it turned out?

Port: Oh, man. I don’t like that question. It’s like…

Waegner: Who’s your favorite child?

Port: The way that I tend to work anyway, and I think this meshed well creatively with how Jon Watts liked to work, is I like to look at whole sequences together and see what kind of pops out, and those are the worst things when you want to tackle those, and then you kind of just smooth it out. The worst thing that I want at the end of the day is to have one sequence or a lot of groups of shots that we really focused on a lot and then have just ones that we’re like, ‘Oh, I think they worked on that a day maybe.’ It looks like, just really bad or something, so I try to just, ‘Do what you can in the timeframe, and just keep going in layers and layers and smoothing it out.’ The most important thing is, does it take you out of the movie? Does it make you think, ‘Whoa, that’s kind of wonky animation, or the lighting looks off, or something is taking me out of the thing.’ A lot of times, that’s like, how fast is the shot, or things like that are probably more forgiving than a very long shot, for example. But these are the things that I look for. I was very proud of all the sequences. I think since these two guys are here, I’ll mention their scenes. 

The High Bridge sequence was very cool. It changed a lot, as did the end battle sequence… well, for that matter, all of the sequences, but what was very difficult and challenging, visual effects-wise, with the High Bridge sequence, is just how much live-action integration with cars and effects and digital environments, and indoor and outdoor shots. It’s just a lot. It was probably a couple of weeks on stage and exterior sets, and it was just a lot of very labor-intensive work in order to have that happen. And then with the end battle, I think that was just amazing, given that it was an isolated thing, but they were able to make that work with what we ended up shooting, which was just a couple layers of scaffolding. That was literally all we had, and everything else was just full digital environment. We had a little bit of piece of the shield, and a little bit of the base of the statue, but those got overused really quickly, so we had to use different pieces of them and mix it around.

No Way Home
Final comp shot (Digital Domain/Sony)

BTL: Looking at all the great VFX movies from last year, the people who work in visual effects probably see them differently from laymen, so what were some of the VFX movies you liked, and they don’t necessarily have to be one of the other Oscar nominees? I mean, I personally loved the VFX in Godzilla vs. Kong, for instance.

Edelstein: That is a good question. I think what’s cool is that we went to the bake-off this year, and we were just watching all of the presentations. Even then, I had no idea who was gonna get nominated. There was so much amazing work from all these different shows and people and artists, and all good in its own way. Like you said, Godzilla vs. Kong was huge, right? It’s a giant monkey and a huge lizard, there’s water, they’re in the middle of the earth. All this crazy stuff. Matrix had some amazing stuff that they were doing with the camera rigs that they built, some really crazy technical stuff that they did on that show. I think everybody is just so deserving. There’s just so much great work that everybody did. How anybody chose… I mean, I obviously voted for us

Waegner: I think it was pretty humbling when you go to the bake-off and you see all the films that are there, and what a struggle it was for our individual teams to make a movie like this, at this level, during a pandemic. And then you see all the other films and there’s just so much great, great work that was cranked out during this time, which is really a testament to all the artists who were able to pull it off [while] working from home. I mean, it’s just as simple as that.

Port: A lot of good stuff.

Spider-Man: No Way Home is still playing in movie theaters nationwide, and it will be coming to VOD on March 22 before being released on DVD and Blu-ray on April 12. In the meantime, you can watch a behind-the-scenes VFX reel below featuring the work done by our talented trio of interviewees.


Edward Douglas
Edward Douglas
Edward Douglas has written about movies for print and the internet for over 20 years, specializing in box office analysis, reviews, and interviews. Currently, he writes features for Below the Line and Above the Line, acting as Associate Editor for the former and Interim Editor for the latter.
- Advertisment -


Vicon Introduces Mobile Mocap at SIGGRAPH

Motion capture systems developer Vicon is previewing a futuristic new “Mobile Mocap” technology at SIGGRAPH 2011 in Vancouver. Moving mocap out of the lab and into the field, Vicon's Mobile Mocap system taps several new technologies, many years in the making. At the heart of Mobile Mocap is a very small lipstick-sized camera that enables less obtrusive, more accurate facial animation data. The new cameras capture 720p (1280X720) footage at 60 frames per second. In addition, a powerful processing unit synchronizes, stores, and wirelessly transmits the data, all in a tiny wearable design.

Beowulf and 3-D