Resolve

Between Grade and VFX

I recently worked on the color grade for the feature film The Runaways. There were a few areas where changes were discussed with the client that went beyond your typical color grading tasks and fall more in the realm of visual effects. I had the opportunity to work with advanced VFX software to work out fixes for three of these.

Aging an Actor

The most fascinating of these was the need to age one of the actors in several shots as they were filmed several years before the rest of the coverage and this age gap wasn't part of the story. I did some reasearch online on the changes to the human face and experimented with these. 

 

The changes were made with SilhouetteFX and Resolve. In Silhouette a pin-warp node helped elongate the face, pinch the nose and open the eyes. The face was tracked with a planar tracker linked to the pin warp to make sure the warp effect was honoring face movements.

 

(pin warp and overall nodegraph in SilhouetteFX)

(node graph in Resolve putting all the external mattes together)

In addition the SilhouetteFX roto tools were used to create external mattes for all the key sections of the face, including lips, eyeshadows, cheeks, and forehead so they could be re-colored separately and adjust the make-up to be more in line with the rest of the shots - specifically in the eye shadow and the lips.

And finally, the SilhouetteFX paint node along with a Mocha tracker and auto-paint was used to remove all the acne in the face to avoid having to soften the image too much to smoothen the skin for an older appearance.

In the process I got to learn about the SilhouetteFX software which has some really amazing features. The paint node is the only clone brush I have found that allows for complex inter-frame painting, cloning from neighboring frames and also grading and warping the clone source of a neighboring face while matching everything.

Complex Dust Busting

Another set of clips had some lens dust on them. Generally that's not a huge problem, and Resolve has tools to deal with that. Except in this case the dust was smack in the actors face and was moving around with her movement, traversing her neckline, and moving over her mouth and nose, delicate borders where the usual clone & blend just creates a mess without more individualized control.

 

(left actor, in this frame dust spot is center of cheek)

I had tried numerous tools from Resolve Patch to other automated tools, and to paining frame-by-frame in Photoshop. Nothing would really be satisfactory when the spot moved over the boundary areas, or was creating too obvious a clean-up.

So it was fortuante that I listened to the webinar on the new release of Mocha Pro 2019 and its remove tool. I was familair with the Mocha tracker but not the remove tool. It was clear that this cleanup would require significant cleanplates. And while Silhouette can handle cleanplates and cloning from a single cleanpalte to multiple frames to avoid the boiling of just cloning the frame by frame, it's a very complex node tree that is very manual. Mocha Pro's remove tool makes it every easy to work with numerous clean plates and then blending the frames in between. Out of the 200 some odd frames, I think I ended up sing 40 clean plates and the rest was blended together and automated.

Recoloring Hair

The last of the problems involved some hair coloring. There were several clips where the hair color was off and too dull, mostly due to lighting on set. But in one scene the actress' hair color is part of the dialog, so it was important to get it closer. The attempts to to key the hair were unsatisfactory because there were too many similar tones in the frame. And constraining the key with a tracked power window in Resolve was still not getting me quite there.

Since I had just worked with Mocha Pro 2019 on the remove tool, I created an external matte for the hair that was much more accurate. By using multiple tracked rotos for the hair and bouncing locks, while also exclusing the hair band, making full use of the planar nature Mocha, I got a detailed matte within a reasonable amount of time.

 

Screenshot of Mocah Pro drawing the different roto masks.

In the end it takes a lot of tools to work together to make this all happen. A fun but very time consuming endavour. And a reminder that while 'fix it in post' is often possible, a few minutes on set to fix something can spare hours in post to work around it. Fixing it on set is always the preferred option, unless it's literally impossible or it means not getting the shot at all due to other constraints.

Custom ACES IDT for Resolve

Note that DCTL scripts require the studio version of Resolve.

I was just involved in a discussion on creating IDTs for color grading in ACES cct in Resolve 14 when there is no built-in IDT. Or if the existing IDT isn't ideal.

Here's the process that I worked out: 

When there is no IDT specified in the project and the clip, then Resolve expects the data to be in linear gamma and AP0 gamut. An input LUT can be applied before Resolve translates the clip to ACES cct color space. Thus a LUT that can translate the camera data into linear / AP0 is a proper replacement for an IDT.

This can easily be achieved in LUTCalc (https://cameramanben.github.io/LUTCalc/). To test this theory I took a clip and set the default Slog3/S-Gamut3.Cine IDT and took a screen grab for reference. I then created a custom LUT with these settings in LUTCalc. I set the clip's IDT to None and added the new LUT as the input LUT for the clip.

As seen below, the resulting image matches the reference images from the original IDT, suggesting that the two operations are equivalent.

This now opens the opportunity to make additional customizations to this input LUT to taste, or create a LUT that matches the camera specifics of unqiue cameras.

Based on this article, the other option is to create a DCTL script: http://acescentral.com/t/adding-idts-to-resolve/161/2. A DCTL script has the advantage that it's precise math rather than interpolated lookup table. The code in a DCTL script matches the math precision the built-in IDT use.

Using the Sony SLog-3 IDT and converting it to a DCTL file, which then is placed into the LUT folder and used instead of IDT or Input LUT also creates an equivalent image. In fact it creates an exact match when using a reference wipe, whereas the input LUT yields minor variations, presumably based on the less precise math or LUTCalc having slightly different input values.

Note that DCTL scripts require the studio version of Resolve.

// SLog3 / S-Gamut3 DCTL for ACES IDT replacement
__CONSTANT__ float color_xform[9] =
{
   0.6387886672f,  0.2723514337f,  0.0888598991f,
  -0.0039159060f,  1.0880732309f, -0.0841573249f,
  -0.0299072021f, -0.0264325799f,  1.0563397820f
};

__DEVICE__ float slog3_to_linear(float v) {
  float result;

  if(v >= 171.2102946929f / 1023.0f)
  {
    result = _powf(10.0f,(v*1023.0f-420.0f)/261.5f)*(0.18f+0.01f)-0.01f;
  }
  else
  {
    result = (v*1023.0f-95.0f)*0.01125000f/(171.2102946929f-95.0f);
  }

  return result;
}

__DEVICE__ float3 transform(int p_Width, int p_Height, int p_X, int p_Y, float p_R, float p_G, float p_B)
{
  // Convert from SLog3 to Linear
  float3 linear;

  linear.x = slog3_to_linear(p_R);
  linear.y = slog3_to_linear(p_G);
  linear.z = slog3_to_linear(p_B);

  // Convert from S-Gamut3 to AP0
  float3 aces;

  aces.x = color_xform[0]*linear.x + color_xform[1]*linear.y + color_xform[2]*linear.z;
  aces.y = color_xform[3]*linear.x + color_xform[4]*linear.y + color_xform[5]*linear.z;
  aces.z = color_xform[6]*linear.x + color_xform[7]*linear.y + color_xform[8]*linear.z;

  return aces;
}

Here is the same frame with all three different methods: the built-in IDT, the input LUT, and the DCTL script.