A couple of days ago I posted a breakdown of a rig removal shot that I did on the Angelina Jolie movie Salt. I got a comment on Vimeo from Rob Antill asking for more details about how I did the frame-by-frame paint work. I wrote a pretty detailed explanation on Vimeo so I thought I should copy it here.
Check out the shot first:
In a rig removal shot you ideally want to clean up one frame and then use that frame as a patch to fix the other frames in the shot. You can create more than one patch for a shot and mix between the clean frames if the shot changes too much for one patch too work. There are some situations that would need so many different patches you’re better off just breaking out the clone tool and painting up each frame. Usually you can combine the two techniques. I this example we used patches for the roof of the truck but painted everything in front of Angelina.
I use Silhouette for any hardcore frame-by-frame painting that I do. It has similar paint tools to Photoshop and allows you to clone from different frames. I use a Wacom tablet all the time at work and this is particularly useful for paint work.
My workflow would be something like this:
* degrain the plate before loading it into Silhouette
* pick a hero frame, one with least motion blur and that’s easy to clean up
* clean it up with the clone tool, using other frames if necessary
* use this frame as the source to then clone from to clean up other frames around it
* render it out and re grain in another package
There is a lot of back and forth tweaking frames. Each frame can look great on it’s own but the sequence can look horrible when you play it back if they are all different.
I wouldn’t do the above for the whole frame at once, I’d concentrate particular areas at a time, get that area working for the whole shot, then go back and do another area. Where the lighting changes a lot you might have to create two hero frames at extremes and then use transparent brush strokes to mix between them as you go.
Silhouette has great tools for making it easy to line up the frame you are cloning from with the frame you are painting to. You can overlay the source frame and translate, rotate, scale and skew it to fit to the destination. Make an effort to learn all of the keyboard shortcuts for this as it will speed you up dramatically.
You can also use the colour correction brushes in Silhouette if the lighting changes in the areas you are cloning and your source doesn’t quite match the colours in the destination. I’ve found you need to be subtle with them so I’ll use very small values in the settings an apply them with a very soft brush stroke.
When I’ve finished the paint work I’ll render it out, take it into Shake or Nuke, and do a difference key between my before and after degrained plates so I end up with just little patches of your paint work. Then I’ll add grain to this small patches and composite this patches back over your original grained plate. This way you have change the smallest amount of the original plate as possible.
The big advantage Silhouette has over other apps with paint tools, like Nuke or After Effects, is that it bakes down your paint work to a cache on your disk, rather than keeping the paint strokes as editable vectors. This makes the process much much faster. It also makes it much easier to clone your paint work from one frame to another which helps prevent the boiling you can get if your paint strokes are different in each frame. You do loose some flexibitly this way, if you have to change your plate you’re pretty much stuffed and have to start again but it’s one of those trade-offs.
In Nuke you can clone from the plate in other frames but you can’t clone your paint work from other frames easily. You can do it in After Effects but it gets slow really quickly and I’ve seen it get stuck in never ending loops if you try to clone strokes from one frame to another and back again.
EDIT: If you liked this post check out my Top 10 Roto Tips.
EDIT: I have added a new post about using these techniques in stereo: here.
Nice and accurate, my congrats.
Thanks 🙂
Thanks 🙂
You actually worked on Salt and did that? I’m sorry for the awe but I’m very new to all this and thinking that you did THAT on such a high end movie humbles me to levels previously unknown to me. That out of the way, AWESOME JOB! I read your tips on roto and will put them to use very soon. You can take a look at my first (sloppy) roto job on the video Kat Killa. Thanks for taking the time to post your tips and help the rest of us mortal. Cheers.
I’m glad you found my tips useful. You don’t need to be in awe. It took me a while before I got to work on feature films. Did you see the music video I did at university? I’ve come a long way from there. Everyone has to start somewhere. It’s good that you are making your own stuff.
Awesome job! Thanks for the tips.
Impressive work on a great film! I’m just now getting deeper into the software research on roto and paint tools. Silhouette looks to be quite powerful. In comparison, I also found this video on the mocha Pro Remove module to be pretty sweet: http://www.imagineersystems.com/videos/mocha-pro-remove-overview-fix-it-in-post/view
I’m interested in your professional thoughts on Silhouette vs. mocha Pro..?
Thanks!
I love tracking stuff in Mocha. I use it regularly for clean up work, but only for tracking. I then send the tracking data to Nuke to complete the work.
We have a slightly older version of Mocha at work so I haven’t had a chance to try the remove tool but I watched the video in the link, and I’ve been to a product launch presentation. None of the shots that I’ve seen them use for demos are hard. None of them would take that long to do in Nuke, especially if you have the tracking data from Mocha. I don’t know how much control Mocha gives you but there are loads of tools in Nuke to help you match lighting and colour changes and there is the built in paint tools so you don’t have to use Photoshop to create clean plates. There are also loads of other tools to help you when the simple approach doesn’t work.
I also haven’t seen the results of the mocha shots up close, only in online videos and in presentations. The paint and roto guys at Framestore have to work to really high standards here so without seeing the results I wouldn’t like to comment on the quality.
I’m sure in the cases that it works the results would be fine and it would be a bit quicker than using Nuke but there are many many shot where it wouldn’t stand a chance. Because we already had Nuke and a version of Mocha we could track with we decided it wasn’t worth upgrading to the Pro version just for the remove module.
The method of tracking in clean plates is one of the best ways of cleaning up plates if the shots allows it. But with this Salt shot there is so much movement and lighting change I would have had to created clean plate for every other frame and it was almost impossible to get any tracking data that I could have used. This meant that I was better off painting each frame individually. This is the only time I would use the Silhouette paint tools but it is also the kind of situation they are built for. You could just about use the same approach in Nuke but it would be painful and slow.
As for roto, I’ve only used Silhouette for hardcore roto shots and I think the tools are great. The few times I’ve tried it in Mocha I didn’t get it. That might be because I didn’t know all the little keyboard shortcuts and stuff that I knew in Silhouette. I know the Mocha tracking is really helpful for roto but the workflow and the controls for the little details are great in Silhouette.
Dang Conrad… you really know your stuff! I can see that having knowledge in both Silhouette and mocha would be equally useful and perhaps even required based on the goals within the shot environment.
Thanks for the fast and very detailed response. In fact, there was so much great information in your reply that I had to read through it a few times just to absorb it all! 🙂
Don’t forget that the next version of Nuke is planar tracking in it. Hopefully it is as good as Mocha’s. Then you could do without Mocha and just use Nuke and Silhouette.
Good point. Efficiency-wise it not only makes sense to streamline the day-to-day workflow, but to also reduce the amount of software package use as well. It’s tough to be fully knowledgeable and derive the full benefits of any single package if you are jumping back and forth. From Twitter it sounds like Nuke 6.3 could arrive any day now.
Hi! great job on a not-so easy shot !
But in fact my comment is not only to congratulate you …
i have question i hope you can help me with
I am a Flame artist and I’ve been exclusivity so for more than 7 years now
most of my work was advertising and documentaries but this is about to change
since my next project is going to be a big budget movie – in middle east 30 milion $ is really really big!- i will be the lead compositor and we are discussing the best ways
to use Nuke extensively in our pipeline (the guys here used to depend on AE as a back room compositor or as a support system for Flame)
personally i like nuke even though i didn’t use it much may be because i never faced a challenge that Flame couldn’t handle -and i had my share of really challenging shots- so what do you think is the best way to get the Flame/Nuke combination work?
or should i go with my previous suggestion and add a Flare seat alongside the
existing AE seats ?
Thanks
Hi Joe,
I’m afraid that I don’t have much experience with Flame. From what I’ve heard it does similar stuff to Nuke when it comes to compositing. I know the commercials department a Framestore used Nuke along side their Flames suites.
I guess it depends what kinds of work you are going to be doing. If it can all be done in After Effects and the artists are comfortable using that then I don’t see why you’d need to switch but if you think that you are going to go beyond the capabilities of After Effects then Nuke would be the next logical choice. I don’t know much about Flare or how it compares to Nuke.
Sorry I can’t be much more help.
Hi Conrad,
Thinks for sharing your techniques. Shot looks really touch. I do stereo paint but at home i used to practice wire/rig removal shots too.
I don’t know Silhouette. Previously i used to use After Effects for paint work but for last 5 months i am using Photoshop as its much better then After Effects in terms of frame by frame paint.
What you think Silhouette has advantage over Photoshop in terms of paint?
I have never used the video features in Photoshop so I don’t know what it is like to work with. I know some of the prep team at MPC used it and preferred it over Silhouette. I am usually working in Linux machines so Photoshop doesn’t work.
I like the speed of Silhouette but I don’t know how it compares to Photoshop. I have also been told that you have to be careful with colour spaces in Photoshop but I don’t know for sure.
Hi Conred,
Firstly thank you for you answers n second thing that now i can tell you why some guy prefer photoshop over silhouette as i do because in photoshop one can manually cut specific area of a frame and past it to the another frame in the sequence and can adjust it using warp, smudge, scale and other transformers available in photoshop which one cant do in silhouette.
..Best to my knowledge.
Please let me know if reveal tool in the rotopaint node of nuke, can work like sequence paint we do in other softwares. can i connect rotopaint node to itself as a bg, so i can paint over my previous paint strokes. (on single rotopaint node..
or if you can tell me how can i paint over my previous paint strokes in the same rotopaint node, i need to take reference from other frames same as we do in silhouette..
i hope you got my point… i GOOGLE but 🙁
Thanks. vik
Hi Vikas,
I can definitely see why people would like to use Photoshop to make patches. I do exactly the same thing, but in Nuke. That way I can use the tracking tools, I can keyframe animate the transforms and warps, and I can use the 3D environment to project the clean patches on geometry.
Using clean areas from one frame is always easier than painting something on every frame.
As for painting in Nuke, you cannot clone paintwork that you have done on one frame, onto another frame. It is a limitation of vector based approach that Nuke uses. You can create a second paint node below and then clone your paint work, but I find this is really slow, not very flexible, and the onion skinning never works properly.
The paint tools in Nuke are fine for creating a single clean plate to track on but I would never have attempted to paint something like this Salt shot in Nuke.
“Eh, we’ll just remove the GIANT GODDAMN CABLE in front of Angelina Jolie in post.”
Hi Conrad really like your post is one of a few places on the web that explains frame by frame painting when punched into gogle. I did try your advice but still got the boiling effect even thought each frame looked ok so ended up patching.I guess I’ve not got the hang of the brushes yet or the grading brush. If you have time would you explain the roll-mixing technique ? I ve search the net and not found any thing helpful.
your right about photoshop once you work in 32 bit its basically crippled with all the paint features turned off.
Thanks
I do a lot of compositing in Nuke, but because I find its paint interface so horrible to work with, I do all my paint work and asset creation in AE. You can render out paint work from AE as a TIF sequence with its own alpha channel included; this would save you having to do a difference key as you’re already starting with an exact alpha of your paint work. I haven’t used Silhouette, but I imagine you could render the same sort of sequence out? Adding the paint over your comp in Nuke is just a simple Merge Over node, and doing color corrections or adding grain then becomes amazingly easy and fast with that included alpha channel.
I never have a problem with AE bogging down because I never clone my own paint strokes as part of my workflow. I also never use the layer I’m actually painting on as a clone source. (This will make more sense in a second.) I choose hero frames to clone from on a copy of my degrained sequence, or paint a clean patch, bake it out, and bring it in to paint from. No more neverending loops. The layer structure usually looks like this:
Layer 1: Degrained Sequence (to paint on only)
Layer 2: Degrained Sequence copy (to clone from)
Layer 3: Baked-out cleanplate patch (to clone from)
Once the paintwork is done, select “On” for the “Paint on Transparent” option on the Paint effect. This is why you don’t want to clone from Layer 1, your paint layer. Any cloned paint strokes from that layer will disappear when you turn on this option. Choose your hero frames from Layer 2 or 3 and you’ll be fine. This also guarantees that AE won’t bog down, and you get all the extra goodies that vector paint allows, like animated splines. Best of all worlds.
You might want to give the Grid Warp node a try. It allows you to very specifically warp and distort a clean patch across a roto’d area over a range of frames. Basically, what you do in Silhouette, but without paint so it’s very smooth. We used it in my studio for some pretty insane stereoscopic rig removal for an upcoming summer tentpole. Wish I could tell you what it is! Let’s just say that rig removals on a certain iconic character’s VERY intricately-patterned costume would have been utterly impossible without this node. Painting was out of the question. You might really enjoy what Grid Warp can do. It’s awesome.
Thanks for the After Effects tips. Sounds like a decent workflow but I like flexibility of not having to manually bake out the frames that Silhouette gives me.
As you rendering your only your paint work with an alpha, you can do this in more recent version of Silhouette. I think it was introduced in version 3. I had learned my work flow in v2. One reason I would still use a difference key is that I would want a solid matte, no semi transparent edges, to apply my grain with. If you have soft edges in your matte you can sometimes get a doubling or softening of grain at the edges of your patch.
I definitely use the grid warp for patching sometimes, but there are always occasions when even that won’t cut it. I really liked the spline warp too.
hi conrad
i hav a question about studio machines .which OS (mac, win or linux) is so flexible for VFX softwares? and which machine is better to give the exelent graphic support especially for our vfx renders.
thnks
maha teja
Hi Maha
These days there isn’t much difference between different platforms except for the availability of software.
All of the Adobe stuff works on Mac or Windows, but not Linux. Although there are fewer of the compatible graphics cards on Apple machines.
Nuke and Silhouette run on all three platforms.
I think that Maya and 3D S Max run better on Windows or Linux, but I’m not sure about that.
All of the big studios that I’ve worked at run Linux.
At home I used to have a Mac but when that needed upgrading I switched to Windows because the hardware was half the price and I had switched from using Final Cut and iPhoto to Premier and Light room.
I know this isn’t a very satisfying answer but the hardware is all very similar these days.
hi conrad
u said right man.thanks for sharing this.i think its very help ful to every one not only me.
You actually worked on Salt and did that? I’m sorry for the awe but I’m very new to all this and thinking that you did THAT on such a high end movie humbles me to levels previously unknown to me. That out of the way, AWESOME JOB! I read your tips on roto and will put them to use very soon. You can take a look at my first (sloppy) roto job on the video Kat Killa. Thanks for taking the time to post your tips and help the rest of us mortal. Cheers.
hi conrad i am Krish..
good work u done.
i am very proficient in paint [frame by frame] like rig removals, scratch removals and all, and mainly i am Stereo paint artist, now i am jumped into VFX paint i am mean now i am complete PREP artist in local studio, in INDIA. now i want to ask u some question can u please let me know about which companies are using prep work in ADOBE Photoshop, and nuke, and what about this prep work feature can u please tell me?
I have never seen anyone use Photoshop for prep work but I did hear that the guys in the Indian MPC office did use it. I have never used Photoshop for prep work.
Most places will use Nuke too.
Indiankrish can u help me in stereo paint
What help do you need?
Did you see this post Stereo:
Frame-by-Frame Painting in Silhouette | Conrad Olson
http://conradolson.com/stereo-frame-by-frame-painting-in-silhouette
thank you. and can u please let me know about paint artist feature in industry. mostly for prep work using software NUKE right. i am planing to go aboard like singapore, malesia canada, which country is best for this VFX field and also for paint work, please can u explain.
Great job, and very nicely explained.
Just one question, when using Sihoutte’s colour correction brushes, is there a way to have it apply just to existing clone brush strokes as opposed to it affecting both the stroke and the background plate.
That’s a good question. I don’t think you can but I would be interested to know if you find a solution.
Hi Conrad- great job you did on this. you might have answered this already, but how long did you spend working on this shot?
Hi Rebecca,
It took me about 5 weeks but I wasn’t working constantly on it during that time. If I had done it uninterrupted it would have probably taken me a couple of weeks.
Nice work Conrad!
What programs did you use to degrain your shot in?
Thanks!
Thanks Stephanie,
At Framestore they had one or two licenses of Fusion, just to run the Neat Video degrain plug-in. They even had a couple of artists with the task of de-graining plates for everyone else. Image Engine also use Neat Video but have setup an automatic script so I never actually see the plug-in controls. Neat Video has been regarded as the best tool for a long time now.
I really like the new degrain tools in Nuke X. They seem to work really nicely and don’t soften the image too much. If I have to degrain anything myself I just use that.
I recently tried the new degrain/regrain tools in After Effects. They looked pretty good but one part of the process didn’t work in float space, so clamped highlights.
Hey just wanna say great work on that shot right there!
Sorry to bother you but I have 2 questions for you.
1.When you said “do a difference key between my before and after degrained plates so I end up with just little patches of your paint work. Then I’ll add grain to this small patches and composite this patches back over your original grained plate” although it seems straight forward to most people, i don’t exactly get what you meant. Do you mind explaining it in a more detailed manner?
2.What’s the difference between just working on the original plate and working on the de-grained plate? Does it mean its better to clone on the de-grained plate?
Thanks for your time.
Hi Elvin,
I’m going to answer your questions in reverse.
It is important to work on the degrained plate for two reasons. Firstly, cloning softens the image and this can make the grain appear soft and smudgy.
Secondly, if you are cloning from the one frame across several frames you get the same grain pattern on each frame. This can make it look like the grain is frozen in that area.
Both of these issues can really make your paint work stand out. In most paint work techniques you should degrain the image and then re-apply the grain to the area that you have applied paint work.
So when I’m painting in Silhouette I do all of my paint work on the degrained version of the plate and render that out. It is always tempting to just add grain back to that whole plate and leave it as that, but it is important to keep as much of the original plate as you can. Degraining also softens the image so you want to limit it to as small an area as possible.
To make sure that I keep as much of the original plate as possible I always key mix my rendered paint work back with my original plate.
It is possible to render out a matte of your paint work from Silhouette so you could use this to add your paint back over your plate. But if you have soft edges here it and you are re-graining it can lead to soft grain on the edges.
So, what I do to create a hard edge matte of my paint work is this:
When I wrote this post the latest version of Silhouette wouldn’t let you replace the plate that you were working on once you had started so it was important to make sure you had the degrain done before you started. I think the newer versions let you replace the plate and it rebuilds the paint work on the new plate, but I haven’t tested that.
techniques is nice discussed. THANK YOU.
Hello, Conrad! Thank you for sharing your technics! I found it very useful for myself.
I have got a question, RotoPaint in Nuke 9 became much faster and stable then it was, is your workflow still actual?
I’m glad you found this useful.
I haven’t actually worked much with Nuke 9 yet. The big VFX companies don’t rush into updating core pieces of software.
I also haven’t had to do such complicated rig-removal shots recently.
I would definitely try to use Nuke’s paint tools, but I imagine they are still not as fast as Silhouette, as Silhouette writes down temp files to your disk as you work and also has many shortcut keys just for the paint tools.
Thank you for you answer! To be honest, I have not worked much with Nuke 9 too, generally still Nuke 8. I have had a wire-removal shot recently and I have used Nuke 8 for it. I have found it a bit unstable and slow, it is a reason why I have asked you . So, I sure, I need to look at Silhouette. Thank you, again.
hi, Conrad!
i am new to paint work ,doing my first paint shot in silhoutee by frame by frame.
getting jitters how to clear it any tips.?
It’s hard. You have to be very careful to line up one frame with the next using the onion skin tool.
If the shot is slow you might be better to create a patch and track it in with Nuke rather than paint every frame separately. But it’s hard to tell without seeing your shot.
hi conrad!
i am new to sequence painting in silhouette ,how to paint (RGB) channel wise with brush
or any other techniques
i am Removing red track markers in green mate.(Getting problems in red channel).