Stereo Frame-by-Frame Painting in Silhouette

My post about frame-by-frame painting in Silhouette is the most popular page on this website so I thought I’d write an update based on my recent experience using those techniques on a stereoscopic shot.

Unfortunately I can’t show the work I was doing yet and I don’t have any stereoscopic footage that I can use as an example so I’m afraid this is going to be words only. I use the terms ‘view’, ‘plate’ and ‘eye’ interchangeably throughout the piece.

I am currently working on Jack The Giant Killer, which was shot in stereo. The only other native stereo show that I have worked on until now was Avatar and most of the work I did then was roto. I did have to paint out a couple of tracking markers but it was relatively easy compare to this recent shot.

When doing clean up work in stereo you have to make sure that whatever fixes you do work correctly and consistently between both plates. It isn’t just important to make sure that the fix looks the same but that it is aligned correctly with the convergence of the shot. If your paint work is relatively positioned even slightly differently between views you will create an area that sits at a different depth to everything around it when viewed in stereo.

Silhouette has a solid stereoscopic workflow for both paint and roto. You can paint in one eye at a time or, if you view both images in the split screen view you can paint in both eyes at the same time. It is easy to toggle between the left, right or split screen view using SHIFT+1/2/3

Obviously the area that you want to paint will usually be in a slightly different position realtively in each plate due to the stereo effect. You need to use the ‘Stereoscopic Align’ view to line up the area between the two plates. This mode overlays the two plates with an emboss effect. Areas that are aligned will appear grey. You can drag in the viewer to shift the images horizontally. Press SHIFT+4 to enter the align view. Once the area is aligned you can leave the mode and paint on the same point in both eyes at the same time. The alignement doesn’t actually move the image but it offsets the position of your paint brushes.

You will need to adjust the alignment between frames if the object that you are painting changes depth during the shot.

By default the clone brush will clone from the same eye that you are painting on so when you paint in the split screen mode you are clone the left eye to the left eye and the right eye to the right eye with identical strokes. This works well if your clone source is at the same depth as the area you are painting. If you are cloning from a different depth however, you could start to have problems as the stereo divergence will mean that the area will be offset differently in each eye.

When I was painting out tracking markers in Avatar the markers were obviously at the same depth as the surface it was stuck to. Once I aligned the views I could paint both eyes at the same time as quickly as it would have taken me to do just the one.

The shot I did this week was much harder. I was painting out a wire that was supporting an actor. The wire was a meter or so in front of the set so it was covering a slightly different area of the set in each eye. If I aligned the plates based on the rope and tried to paint on both eyes at the same time I would be cloning from different areas of the background in each view. If I aligned the plates based on the background the rope would be in different positions relatively and when I painted it in one eye I would miss it in the other eye.

Initially I decided to just do the paint work in one view and then take that view into Nuke and offset the clean plate to the other eye. I did several regions like this but then I got to some more fiddly areas so I decide that it would be just as quick to paint both views in Silhouette.

In some frames the offset of the rope was so big that I could actually see all of the background that was missing in one plate in the other. I could align the background and then use the ‘view’ drop-down in the clone controls to force the paint brush to clone from one plate and onto the other. This was the best option where posible.

If I couldn’t see enough of the clean area in either plate I would clean up one view first and then clone from the clean view to the second view. This isn’t ideal as it caused extra softening but I tired to minimise the number of paint strokes I used to reduce the impact.

I used the ten brush presets in the clone tool to setup brushes so that some always cloned from the left or right and some others were set to auto. Once I got into the flow it was relatively quick to toggle between the brushes and use the keyboard shortcuts to switch between the views.

Frame-by-frame painting should always be a last resort technique for rig removal but until I tried it I thought it would be almost impossible in stereo but it wasn’t as painful as I thought it might have been.