Marvel Studios’ latest blockbuster, Shang-Chi and the Legend of the Ten Rings, directed by Destin Daniel Cretton, has been wowing moviegoers in theaters for a little over a month, and there are many reasons for that.
A lot of Marvel fans have taken to the film’s magical fantasy elements, which required a lot of specialized visual effects, things like the rings of the title, which we first see in use in an opening montage that introduces Tony Leung’s crime lord Wenwu. We also see him use them during his first encounter with Fala Chen‘s Xiang Li in the magical land of Ta Lo before the two fall in love, settle down, and have a son named Shang-Chi, who is played by Simu Liu when he’s in his 20s.
Their first meeting is a fantastic bit of movie magic that was enhanced by the work of Scanline VFX, one of a number of visual effects houses that worked on Marvel’s latest superhero masterwork. Scanline also dealt with the magical bamboo forest that surrounds Ta Lo, both at the beginning of the movie and a scene that leads into the film’s climactic third act battle in Ta Lo.
Micah Gallagher was the Compositing Supervisor at Scanline who basically got all the work from the different departments and matched those assets and elements with the stuff filmed on sets with the actors. Below the Line spoke to Gallagher a few weeks back about what went into creating some of Scanline’s impressive VFX work for the movie.
Below the Line: Before we start talking about Shang-Chi, how long have you been at Scanline, and what was your background before then? Did you go to school to learn VFX?
Micah Gallagher: I’ve been at Scanline for almost seven years in the Vancouver office. Before that, I lived and worked in Los Angeles. I worked at a couple of different companies — Pixomondo, a company called The Syndicate, and actually, I went to school for engineering, and then found myself needing a job out of college and wanted to get into film, and ended up working at a company called The Syndicate. I was just working in their vault, doing shipping and delivering film for color correction. On the floor above me, they were doing visual effects for The Mummy 3 and Spider-Man and stuff like that. I ended up learning stuff from the people upstairs and eventually got into roto paintwork and worked my way up to compositing. That was like 2008 or so. I think that’s when I started, and I’ve just been progressing on the compositing path, which led me to Scanline where I ended up after a couple of projects at Scanline, I became a Comp Supervisor. I was the comp supervisor on Shang-Chi. I now am an Associate Visual Effects Supervisor. After that movie, I got a little promotion, but that’s where things are at with me right now.
BTL: I’m not sure I’ve spoken to a Digital Compositor yet, although in my various conversations with VFX Supes, that role is constantly mentioned. I have a general idea of what it entails, but maybe you can tell me definitively what one does.
Gallagher: Absolutely. There’s so many departments that have to work together to put a movie together, and compositing is the last department that basically assembles everything that has been created, whether it’s a digital creature, or some type of environment, and maybe that’s coming from a 3d artist, or maybe a matte painter has created something. They would handle those elements, or maybe an effects artist who has created an explosion. They would hand those elements to a compositor, and then the compositor, they are usually dealing with the actual footage that was shot and pulling keys on green screens or blue screens and extracting the characters or whatever objects you need from what they filmed, and then putting the CG environment in there and making it all look like it was shot with the same camera. You’re assembling everything, and then you’re putting the final touches on it to make it look as realistic as possible, based on the direction of the directors and the clients and the VFX Supe on the client side.
BTL: And then the lighting department is involved with the final lighting on those shots?
Gallagher: That’s kind of the step before comp. You lay things out, and then once you figure out where everything’s gonna go then you’ve got the departments that are texturing and shading and getting everything looking real and then the lighters are matching the lighting to what was shot and handing it off to the compositors. Compositing works very closely with lighting, more so than any other department. Once you get through lighting, then it’s kind of the last thing to do is comp it, get it into the shot and make it look nice.
BTL: I was reading that in Shang-Chi, Scanline mainly created stuff at the beginning of the movie, the Ta Lo bamboo forest, and then the “dance fight” that follows at the beginning. I’m sure Scanline has done other Marvel movies, but how did you get involved with this one?
Gallagher: Yeah, we have a good relationship with Marvel. We’ve worked on many, many Marvel movies, and we’ve worked with the VFX Supervisor Chris Townsend on multiple projects, so I think it was a good fit. It was just a matter of figuring out what’s the best sequence that Scanline can work on? That opening sequence was just a really fun sequence to work on, when Wenwu meets Xiang Li. Even just the first approach to the forest, when the Jeep crashes and all that, we did that stuff, and then when they actually enter the forest or enter that little clearing and meet each other, we had to recreate this bamboo forest, which was a challenge with just the number of trees and leaves and everything that we had to simulate.
I think the challenge with that sequence is it’s a fight, but it’s a moment [in which] they’re falling in love. That’s something that was conveyed by the filmmakers that they want it to be aggressive and violent, but not go too far. Because we want to show that they’re actually making some connection, as they each use their own extraordinary power they have. Initially, there was this sort of Chi energy that Xiang Li had, and it was this ethereal mist kind of thing that would move around with her movements. It was beautiful, but it didn’t end up really being what the filmmakers wanted to tell the story. We ended up taking that out, and then we just had these leaves and the dust moving around, and that showed what her power was. And then, with Wenwu, this is the first time you get to see his rings and what the rings can do. And so, we were an integral part of developing what those rings were going to look like, once they sort of charged up. We had to figure out, “Okay, what does it look like when they charge up? What does it look like when they charge down? What does it look like when he’s using them to block a punch, or if he’s doing some type of punch himself?” We had to establish the language of the rings, and even just the exact color of his blue rings. We had to dial that in after several [notes of] “Oh, that’s slightly too purple. Oh, that’s slightly too cyan.” It gets really down to the exact red-green-blue values when it comes to that. That was a great sequence. We referenced Crouching Tiger Hidden Dragon, that was kind of the vibe that the filmmakers were trying to… it was kind of an homage to that film.
We also did the Moongate arrival sequence, which was when Shang-Chi and Number 4 and Katy, they’re all in the Razorfirst vehicle, and they travel through the bamboo forest, and then it starts closing in on them. That was one of the most challenging sequences that we’ve worked on in a while at Scanline, just with the billions of leaves in this forest that are all dynamically moving. Each leaf is ray tracing the environment and reflecting as it’s moving. You’ve got millions of trees with billions of leaves. It was just a computational nightmare for us to get all that stuff rendered, but we figured it out, and it ended up being a really exciting sequence, as the force is crashing in on them, so it was two really fun sequences to work on.
BTL: I have a couple questions based on those things. First of all, did they have an actual location for the bamboo forest from which you worked?
Gallagher: They had a set, actually. They filmed all of that in a studio — at least the fight sequence they filmed in a studio — and they basically had walls of bamboo trees, just like a single wall of it, that could be moved around. They always had some bamboo back there, and then we had to extend that as far as you could see. We basically recreated the set digitally, and then just continued building out as far as you can see with bamboo, and then just figured out where’s the sun going to be? How is that going to look as it comes through the forest? They had some really powerful lights on set, but on a lot of the shots, there would be a whole panel of lights. So you’re like, “Okay, which one of those do we want to make the sun?” That was all on a set. When we’re actually going through the forest though, I can’t remember where they filmed it, but it was in a forest, but it was like a pine tree forest. It helped in a certain way with some things, but really, we ended up making most of those shots fully digital, other than the characters in the car — they were in a studio — so we could put whatever we wanted outside the car. Any of the wide shots are pretty much fully digital. We would match-move the vehicles in what they shot, and then, we would create our own assets of those vehicles. At times, as a compositor, I would encourage the team to try and steal what we can from the real car, but for the most part, you’ve got shadows of pine trees on them, and we needed to go fully digital, just because of the changes in lighting direction once we had our own forest in there.
BTL: Did you actually LiDAR or scan the car to have a digital version of it?
Gallagher: Yeah, we had LiDAR of pretty much all the vehicles, and even some of the topography of the forest we would use as a guide to how the ground slopes and maybe little bumps on the ground as the cars are rolling along, so we can replicate that with our digital vehicle.
BTL: For the dance fight, Destin had the two leads, and then you had to do some digital doubling of them as well for some shots?
Gallagher: I think we had around 35 or so shots where we had to replace their faces, because it was a stuntman and woman who were doing the action. That’s always a challenge. They actually would film the stunt people doing a few takes, and then they would film the actors doing it, and then they would select the stunt people’s take, and then they would give us the footage of the actors’ take, so we can try to use some of that. The lighting is kind of the same on their faces, and sometimes we were able to just replace their face with what they filmed. Other times, we had to go with a full CG face or sometimes, it was kind of a hybrid where we might take a still frame or a stabilized plate of one of the actor’s faces and project it onto some geometry onto the stunt person’s face, and so it looks like them.
That’s always a challenge, because it’s something that is one of the hardest things to do in visual effects is convincingly create a digital face. That’s where everyone’s looking, and it’s easy for the human brain to spot something that’s not quite real, especially when you’re looking at eyes. We definitely try to use photographed eyes, as much as possible. There’s a lot of different methods of doing face replacements, and ours is to match move the stunt person’s face, so we get the exact subtle movements that are happening. And then let’s take the geometry of the actors and replace it, so now it’s our actors’ geometry, and then we project our actors’ face onto that, and now it’s stuck onto the stunt person’s face. You might have to do some paint cleanup, to clean up maybe where somebody’s jaw is a little bit wider. I think, the stuntwoman’s face, it was a little bit broader than Xiang Li, the actress and her face, so we had to kind of trim it down a little bit and paint some of the background back in to make sure she had the same face shape.
BTL: It’s really amazing what you’ve been able to do with visual effects these days, but also how seamless it looks, so that most people watching have no idea that’s going on. Sometimes with digital doubles, you can tell it’s CG, but I’m not sure people watching realize you could be spending months and months and months to do a five-minute fight like that one.
Gallagher: Yeah, it took a long time, and also, everything was right in the middle of the pandemic, when the pandemic first started. I think I was on the project for over a year, but it had been going well before that, and Marvel actually paused, filming for a while, because of the pandemic, and then picked it back up when they were able to start filming again.
Also, for us at Scanline, that was right in the middle of us transitioning from working in the office to everybody working from home. At that time, Scanline had about 800 or so employees across all the facilities, and within about a week, we had all 800 working remotely. But we still had to come up with how are we going to maintain the same workflow and output that we were getting when we were in the office? And the surprising thing was, we actually were a little bit more productive, is what we found once we all started working. I think a lot of people were scared that it might be the opposite, but it ended up not being [that].
We’re all still working from home. I think there is a challenge to not being in the same room with people when you’re discussing stuff, and that’s a challenge for the supervisors and the producers. For the artists, if they prefer to just be in their own place and be able to just put their head down and do the work. We have a whole review system that’s similar to like Zoom. It’s something that our owner Stephan [Trojansky] developed. It’s like his own Zoom. It’s called Eyeline, and it allowed us to have meetings, just like we did in the office where everyone has cameras, so you can see. We can talk to people and pull up, review playlists and go through stuff together and annotate and everybody can annotate on and talk. It’s as close as you can get to being in the same room, and helped a lot with that transition.
BTL: I hear all these stories and about all these challenges working in the pandemic, and yet, the work is still pretty amazing. I’m not sure in ten years from now, anyone would know that these movies were made under pandemic protocols at all. Going back, you mentioned creating how the rings were going to look, and I was curious how that worked if you were starting from scratch, but other houses also had to show the rings, so were you sharing those assets and going back and forth on them?
Gallagher: It was kind of a collaborative effort by a lot of facilities. When we first started doing the rings, there were some other studios who were a little further along than us. We were given some of their shots to match to, and then it was kind of like, “Well, we don’t know if we like those anymore.” At one point, it was kind of like, “Nobody knows what we want these things to look like. Everyone, just come up with what you think looks cool.” So, we went back to this Look Development mode, where we’re just trying some different things and some different looks, and, “How bright are they? And how glowy are they? And what color are they?” Until we landed on something.
Our work ended up getting passed to a lot of other facilities, maybe as far as this specific color. I can’t remember if that was something that we actually were the ones that nailed down, or whether we were given a reference for it from somebody else. We definitely had a lot to do with the look of them as they’re being activated. The other challenging thing is later on in the movie, when Wenwu and Shang-Chi fight and they both have their own rings — the orange-yellow rings, and then the blue rings — those fights are supposed to be more violent and aggressive. When they were developing those and the filmmakers were liking those and sending them to us for reference, if we matched it, they were like, “Well, that doesn’t really fit the mood of this scene.” We had to come up with what do they look like when it’s daytime, and these two characters are supposed to be falling in love? It’s almost its own look, but it was still trying to use the same language that’s being used throughout the film. It’s just a little bit more subtle, and I think that’s what separated our sequence from some of the other ones.
BTL: It seems almost like his emotions affect the color of the rings, so when he’s more violent, they change colors from when he’s in a fight with a woman he’s attracted to.
Gallagher: The colors were kind of a fun thing. There’s a moment when Xiang Li takes control of his rings. He shoots his rings out at her, she dodges them, and they go off into the forest, and then he calls them back. And then she uses her Chi power to take control of them, and they change color. It’s like transitioning the power, from his power to her power, so they kind of change to this more yellowish color, which was actually the color of her Chi energy before we ended up getting rid of it. It was a mix of what her dress looked like, it was kind of this greenish-yellow– and so that was showing how the power of the rings can be connected to someone, and it was done through color as to who was connecting with them.
BTL: It sounds like one of those things where you can have lots and lots of meetings between the VFX Supes and Chris, but obviously, everyone has to be on the same page about that aspect of them.
Gallagher: Totally. What color are we going with here? There was a whole language, and there was a PDF document that was like, “Okay, this is the color of when it gets fired, and it’s flying through the air, and then this is the color when it’s a chain, and then this is the color when it’s just a concussive blast. Yeah, there’s just, you really get down, you have to get down to all these little details, in order to make things look consistent throughout the phone.
BTL: Before I let you go, I do want to ask about virtual production, which seems to be something that’s becoming more prevalent with the pandemic, because having live visual effects that you can shoot on set is an interesting prospect. Have you had a chance to do anything in that realm?
Gallagher: We do have a volume capture studio as part of Scanline — it’s called Eyeline Studios, and it’s something that Stefan has developed during the pandemic. I mean, he has been working on it for a while, but it’s really come a long way. We do volume capture where we have somebody who gets photographed with tons of cameras all around them, and then we basically create a digital version of them. What we also can do is put them in an environment where we have LED screens all around them that are projecting the environment they’re supposed to be in with the lighting that they’re supposed to have on them. We are able to actually get that lighting incorporated into the digital version of them. This is something that we used a little bit of it on Shang-Chi, but it was still in the development process. It’s being used a lot more now. If we were where we’re at now, we probably would have used it exclusively for all the face replacements on Shang-Chi, but it wasn’t quite there yet. So we used the other methods that I described, but there were a couple of shots where we were having trouble with facial animation, and they wanted the character to maybe grimace when they’re doing their moves, and all the projections and everything that we were putting on the face, the face was a little bit stoic and it wasn’t animated enough. There were a couple of shots where we actually did go to volume capture, and we had somebody in the volume capture studio, make all these facial expressions, and then we used the asset of Wenwu to recreate those facial expressions, and we’d run that through the camera, and now you’ve got whatever expression you want me and it’s using his mouth and the shape of his mouth and jaw and everything. We did that for a couple of shots where we needed a specific expression. It’s a pretty fascinating thing that’s becoming more and more popular in visual effects for sure.
Shang-Chi and the Legend of Ten Rings is still playing in many theaters nationwide, and it will eventually hit VOD and Disney+ as well. Look for our second feature on the movie’s visual effects with the VFX Supe from Weta Digital shortly.