James Cameron’s “Avatar: The Way of Water” sends the visual effects team back to the technological drawing board to achieve, yet again, breakthroughs never before seen on-screen — underwater performance capture.
With the first “Avatar,” senior VFX supervisor Joe Letteri and VFX supervisor Richard Baneham of Cameron’s Lightstorm Entertainment were part of the brain trust that ushered in an innovative performance-capture system that allowed the “Titanic” director to see CGI characters and environments in real time as if it was live-action filmmaking. Their efforts did not go unrecognized as they went on to win both Oscar and BAFTA awards. This new film sees the duo reuniting to flex familiar muscles, along with the tall task of creating digital water in a 3-D, high dynamic range, high frame rate (48 fps) environment.
“We pushed the facial [performance capture] design as far as we could,” says Letteri, a four-time Academy Award winner. “But we realized for this film that we needed a better understanding about how performance really works.” On “Avatar,” the technology was based on the FACS system (facial action coding system) that combined a head-rig with a single standard-definition camera to record the expressions of the actor whose face was dotted with computer-readable markers. The revolutionary tech found its way onto several blockbuster film sets such as “The Hobbit” trilogy and was refined over the years leading up to Robert Rodriguez’s “Alita: Battle Angel” where it reached its metaphorical end of life.
Letteri reimagined the system for “The Way of Water” with actor performance in mind. “I started to think about how the muscles in the face work. All of our muscles work together to make an expression. You don’t say, ‘move this muscle’. Your brain thinks and your muscles express the right thing. We wanted to give animators a tool that would do something very similar,” he says.
A muscle-based system with a built-in neural network was developed that moved each layer of the face — muscle, tissue, skin — holistically by the performance capture or by animators directly. The change gave animators a deeper understanding of a performance which, in turn, brought more realistic emotions to their Na’vi counterparts. The head rig was also updated with two high-definition cameras providing more shape to the face, higher fidelity and additional information on the motion to tune the final image. “Our actors do such a fine job committing to their characters. It’s our job to protect the integrity of that performance and shepherd it to the screen,” Baneham says.
James Cameron talks with his crew in a studio water tank.
(Mark Fellman/20th Century Studios)
The story picks up from the 2009 film, with Jake (Sam Worthington) and Neytiri (Zoe Saldaña) and their four children — Neteyam (Jamie Flatters), Lo’ak (Britain Dalton), Tuk (Trinity Jo-Li Bliss) and adopted daughter Kiri (Sigourney Weaver). They’re forced to leave the forest and hide out in an oceanic village to escape the humans hunting them down.
Pandora’s enchanting oceans is where Cameron’s ambitious vision (and visual effects) shines. The most lavish creation being the reef village of Awa’atlu, home to the Metkayina clan, led by wife and husband Ronal (Kate Winslet) and Tonowari (Cliff Curtis). Adapted to sea life, they’re considerably different in appearance than their forest counterparts, the Omatikaya. More greenish in color, they have enlarged eyes, bigger chests and fin-like cartilage that protrude from their arms and legs with wider tails to help them swim. Villagers ride winged creatures called ilu (the marine version of the ikran seen in “Avatar”) and spiritually connect with tulkun, whale-like beings that can grow 300-feet long. In the Cove of the Ancestors, their Spirit Tree lies underwater with an otherworldly luminescent glow. But it was more than underwater life that visual effects curated. Crashing waves, moving currents and water splashes had to be digitally created too.
Jake Sully rides a winged creaturs called an ilu.
(20th Century Studios)
The motion-capture system had to be adapted for underwater use as well and the motion-capture suits were adjusted to improve performance capture.
The visceral aquatic sequences were all shot in water (which included simultaneously filming all ocean scenes of upcoming “Avatar” sequels 3, 4 and 5). Two massive tanks were built at Lightstorm studios in Manhattan Beach, one with a 250,000-gallon capacity where the director could film larger action sequences with a wave mover. To capture the actors’ performances, two separate immersive sets akin to giant green screens, called volumes, were constructed, one that was sunk in the water to record the underwater action and another placed above the tank for surface interactions. “We were able to align them geographically and temporally, which allowed us to capture everything above and below the water at the same time,” Baneham says.
Cast, crew and stunt performers all went through extensive training to hold their breath using techniques from free-diving instructor Kirk Krack, so as to limit their air bubbles because the performance capture system was unable to tell the difference between them and the marker dots on the performers’ face and wetsuits.
Sam Worthington dons the upgraded performance-capture technology to shoot “Avatar: The Way of Water.”
(Mark Fellman/20th Century Studios)
The awe of “The Way of Water”’ is in the minutiae. Each creature, plant, tree and, of course, Na’vi, was digitally created. Lighting was approached differently from the original movie, mimicking the true photographic nature of how light reacts in a real-world setting. Even how water rolled off Na’vi skin was looked at with a fine eye as each bead trickled down the tiny hairs creating those familiar trails. With the Metkayina village, a sun-drenched sky provides an optimistic feeling, but as conflict arises, shifting weather patterns pepper the murky mood with darker clouds. The result is visually striking story from Cameron, where again, visual effects has hidden in plain sight despite 9-foot-tall lemur-like creatures appearing on screen.
“You want people to live in the detail and feel the emotion of the characters. The subtext of the characters is where we live and die,” Baneham says. “If we can get the effects to feel real, and the look of them to feel photographic enough, that’s honestly a job well done for us.” Letteri adds. “When you’re working with a team of artists who are all experts in their field, what we all have to be aware of to some degree is how to make a film. That collaboration is fundamental in filmmaking in any aspect whether it’s live-action or what we are doing digitally.”