Frame-by-frame Painting

A couple of days ago I posted a breakdown of a rig removal shot that I did on the Angelina Jolie movie Salt. I got a comment on Vimeo from Rob Antill asking for more details about how I did the frame-by-frame paint work. I wrote a pretty detailed explanation on Vimeo so I thought I should copy it here.

Check out the shot first:

In a rig removal shot you ideally want to clean up one frame and then use that frame as a patch to fix the other frames in the shot. You can create more than one patch for a shot and mix between the clean frames if the shot changes too much for one patch too work. There are some situations that would need so many different patches you’re better off just breaking out the clone tool and painting up each frame. Usually you can combine the two techniques. I this example we used patches for the roof of the truck but painted everything in front of Angelina.

I use Silhouette for any hardcore frame-by-frame painting that I do. It has similar paint tools to Photoshop and allows you to clone from different frames. I use a Wacom tablet all the time at work and this is particularly useful for paint work.

My workflow would be something like this:
* degrain the plate before loading it into Silhouette
* pick a hero frame, one with least motion blur and that’s easy to clean up
* clean it up with the clone tool, using other frames if necessary
* use this frame as the source to then clone from to clean up other frames around it
* render it out and re grain in another package

There is a lot of back and forth tweaking frames. Each frame can look great on it’s own but the sequence can look horrible when you play it back if they are all different.

I wouldn’t do the above for the whole frame at once, I’d concentrate particular areas at a time, get that area working for the whole shot, then go back and do another area. Where the lighting changes a lot you might have to create two hero frames at extremes and then use transparent brush strokes to mix between them as you go.

Silhouette has great tools for making it easy to line up the frame you are cloning from with the frame you are painting to. You can overlay the source frame and translate, rotate, scale and skew it to fit to the destination. Make an effort to learn all of the keyboard shortcuts for this as it will speed you up dramatically.

You can also use the colour correction brushes in Silhouette if the lighting changes in the areas you are cloning and your source doesn’t quite match the colours in the destination. I’ve found you need to be subtle with them so I’ll use very small values in the settings an apply them with a very soft brush stroke.

When I’ve finished the paint work I’ll render it out, take it into Shake or Nuke, and do a difference key between my before and after degrained plates so I end up with just little patches of your paint work. Then I’ll add grain to this small patches and composite this patches back over your original grained plate. This way you have change the smallest amount of the original plate as possible.

The big advantage Silhouette has over other apps with paint tools, like Nuke or After Effects, is that it bakes down your paint work to a cache on your disk, rather than keeping the paint strokes as editable vectors. This makes the process much much faster. It also makes it much easier to clone your paint work from one frame to another which helps prevent the boiling you can get if your paint strokes are different in each frame. You do loose some flexibitly this way, if you have to change your plate you’re pretty much stuffed and have to start again but it’s one of those trade-offs.

In Nuke you can clone from the plate in other frames but you can’t clone your paint work from other frames easily. You can do it in After Effects but it gets slow really quickly and I’ve seen it get stuck in never ending loops if you try to clone strokes from one frame to another and back again.

EDIT: If you liked this post check out my Top 10 Roto Tips.
EDIT: I have added a new post about using these techniques in stereo: here.