Shake and Tremor Intermediate Tutorials

This tutorial picks up at the riveting end of the Beginning Tutorial and adds a bit of complexity. In this week's episode, we will discuss (OK, "discuss" means I'll blather on for hours) color correction, premultiplication, optimization, motion blur, and the Time View, absolutely not in that order.

If you didn't do the first tutorial, you can hit the Load button and read in the script from <ShakeDir>/doc/pix/vp/vanilla1.shk

 

Slipping Time Sync

Before we start, we should read in the other two remaining elements. Do an Image - FileIn and read in <ShakeDir>/doc/pix/vp/bg/cloud.sgi and vp/shadows/shadows.1-56#.iff.

cloud.sgi shadows (sequence)

If you look at the composite at frame 31, you might notice that the smoke happens to be out of sync with Vanilla. For some reason, and don't blame me, the smoke starts one frame later than it should. Honestly, I didn't set this up to be cute:

frame 31

 

Click on the smoke element in the Node View, and then open the Time View. In the Shake interface, this is the last tab in the lower-left pane. In the Tremor-style interface, this the last column in the bottom pane. In the Time View, you can see all your clips laid out in time.

You can select nodes either with the Node View or the Time View. You can also look at nodes, load parameters, or ignore nodes by clicking on the buttons on each clip. You can also load just the active nodes by turning on the Select Group toggle on the bottom left of the Time View.

To slip the sync one frame earlier on the smoke element, just drag it left one frame. The numbers inside the clip represent the clip range that is being read from disk. When you drag the clip, the numbers outside of the clip are the frames (exclusive) between which the clip runs in the composite:

Because we don't want the clip to start too early, drag the left end of the smoke clip on frame to the right so it starts at frame 1 with the others. This indicates that only frames 2 to 56 will be read from disk:

If you go back and look at frame 30 now, you will see the smoke properly synced up:

frame 31

 

Optimization

If you look at the shadows image or place the cursor over the node in the Node View, you will see that its channel type is BW. This means Black and White, meaning there is only one channel - no RGB and no alpha. This will become significant a bit later on, but for now, you should note that because it is only one channel, it takes up less disk space and loads faster than a 3-channel RGB image. Therefore, one optimization tip is, if possible, to save your images that are just black and white as 1 channel images. In the interface, you can do this by applying a Color - Monochrome node to a FileIn and then doing a FileOut as a new image name. No need to do this here, since I've already done that for you. Fancy that. Note that many compositing and paint packages do not support 1 channel images, so test your pipeline before throwing yourself into battle. You can also easily make monochrome images with the command-line:

shake myRGBimage.#.iff -mono -fo myBWimage.#.iff -t startFrame-endFrame -v

If this makes no sense, check the Command-line documentation.

There is another optimization step you should take early on in your compositing process, as it will not only help you in rendering speed, but also when actually building your script. This magical wonderful trick is designating a Domain of Definition. This is a rectangular region around your element telling the software what the important part of the image is, i.e., everything that isn't pure black on the RGBA channels. For example, if you look at the vanilla FileIn node, most of the image is black - we are only interested in Vanilla herself. Therefore, select the vanilla FileIn node by clicking on it and insert a Transform - SetDOD node. "DOD" stands for the aforementioned Domain of Definition.

Make sure the on-screen controls are visible by checking that the On-screen controls are set to Always , and that the DOD display toggle is also set on. This second button merely turns on and off the display of the DOD and the frame border. It has no role in the processing of the image.

You can now grab the corners or the edges and drag them in to frame Vanilla. When you do this, you are telling the software to only be concerned with a smaller portion of the image. The rest of it isn't even loaded in. This will cut down on memory usage, I/O activity, and processing time, both up- and downstream of the SetDOD. Isn't that swell?

vanilla vanilla with DOD

Since the clip is animated, we will have to animate the DOD as well. Turn on the Autokey button in the Viewer . When you click this on, it sets a key. If you screw up, hit the Delete Key button . Going through the 56 frames, put in keys every 10 frames or so. I put in keys at frame 8, 15, 21, 28, 47, 51, 53, and 56. As you place keys in, you will see little key notches in the Time Bar:

To quickly jump to and from these keys, use the Next or Previous Key buttons on the Time Bar, or just use the Up and Down arrows on the keyboard.

Lucky you, you get to do this to two other nodes: smoke and shadows. The good news is that once done, every computation afterwards if more efficient.

For more information, jump to Overview - Domain of Definition.

 

Motion Blur

To look at motion blur, we probably need some motion. In the real Vanilla Pudding shot, a dark cloud shoots quickly in as Vanilla amps up her anger. They have a cool cell animation, but then, I wouldn't be able to talk about motion blur and you'd have to download another 10 Mb of data, so we are going to use the still image cloud.sgi that I painted while testing out the QuickPaint node. My apologies to Wild Brain...

First, hook the clouds into the comp by selecting the Layer3 node and attaching yet another Layer node. Hook the cloud into the second input of Layer5:

Because the Layer nodes are really stacking up, I want to rename Layer5. Make sure its parameters are loaded by clicking on the right side of the node. The very first parameter (top of the list for Shake-style, far left side for Tremor-style) is nodeName:

You can change the name here. You can't use spaces, funny characters, or node types, like Blur, since there is a Blur node. Here, I've named it Over_Cloud, to help me know what operation I'm using and what element it is comped over.

Now to move the cloud. We could use a Move2D or 3D to pan, but I want to use a CornerPin to help with a bit of the perspective. Therefore, click on the cloud node and insert a Transform - CornerPin:

Vanilla's expression changes from curiousity to anger over the frames 30 to 37. We are first going to set the keyframe for the cloud's final position, so go to frame 37, make sure the Viewer autokey button is on , and position the corners. Notice that the corners do not match exactly with the borders of the full video frame, as the cloud image is 500x400 pixels. Anyway, I grab the corners and adjust them, pinching in the bottom corners a bit to give perspective:

frame 37

 

I now jump back to frame 30. I can grab the center crossbar to move all four corners at once, or I can grab an edge to move two corners:

frame 30

 

I therefore now have an animation over seven frames.

The Transform nodes Move2D, Move3D, CornerPin, CameraShake, Pan, Rotate, Scale, Stabilize and MatchMove all have motion blur parameters associated with. There are three parameters - motionBlur, shutterTiming, and shutterOffset, with the second two tucked under a sub-tree:

Ok, not being shy at all here, but the motion blur in Shake and Tremor kicks ass. Rather than render several in-between images and averaging them together, which, let's face it, is pretty lame, we track each pixel through the transformation. Here, I've set the motionBlur in CornerPin1 to 1:

No motion blur motionBlur of 1

The motionBlur parameter controls the quality level. With 0, it is turned off. 1 is the high setting. If you want to get an approximation and decrease your render times, you can drop it down to .5 and still be acceptable, and .1 for a test level:

motionBlur of .5 motionBlur of .1

The shutterTiming parameter specifies what fraction of the frame we expose for the motion blur. For example, .5, the default, means that the movement for only half of the frame is exposed. We use .5 since a film camera has a shutter exposure of around 178 degrees out of 360, so we round it off to .5. If you put it to 0, motion blur is turned off. If you set it to 1, that means the movement over the entire frame is calculated. You can also set this number beyond 1 to calculate later movement into the current frame. Therefore, if you were trying to match a video camera with a shutter setting of 1/300, you could enter 1/300 into shutterTiming and it would resolve that to .0033, or not much blur at all, so why bother anyway?

shutterTiming of .5 shutterTiming of 1

The last parameter is shutterOffset. This controls the beginning point in time, relative to the start of the frame, at which we begin to consider the motion. By default, this is 0, so we calculate the motion from the start of the frame. If you set it to -.5, it would be the motion from the previous half frame of movement:

shutterOffset of 0 shutterOffset of -.5

 

As I mentioned, each node has a setting that you can individually tune. However, what if you want to turn all of them off at once? In the Globals tab is an additional triplet of motion blur settings, with the same names. The first two are multipliers of all the other node motion blur settings, so if you enter 0 in your Global motionBlur, it will internally multiply all other motionBlur settings by 0, turning everything off. If you want to return to full setting, set it back to 1. Typically, you tune your motion blur and then set your Global motionBlur to 0 as you finish the rest of your script. When you are ready to render, you reset your Global motionBlur value to 1.

 

Color Correction with the ColorCorrect node

Now that we've added the cloud in, we can begin to consider the coloring. In Vanilla Pudding, they use strong saturated colors, so we don't want to modify her. However, the hill in the background can be enhanced by boosting up its color to suggest atmosphere. By "atmosphere", I mean the meteorological phenomenon, not the romantic staging practice. We can then animate this down when the cloud passes over it.

Shake and Tremor have a large collection of color correction nodes, based mainly on the command-line inheritance they share. Most of these are single-function nodes in that they only apply a single mathematical operation - adding, multiply, gamma function of color, etc. For Shake 2.4/Tremor 1.0, we've introduced a master color corrector which can duplicate many of these nodes, cleverly named ColorCorrect.

Select the hill node in the Node View and insert Color - ColorCorrect. Typically, you want to color correct before you do your transformations, but it's not a rule you'll get flogged in public for if you break it.

Although you can do lots of tricky clever things with this node, we aren't going to do that. You could tune your RGB channels globally, just in the dark areas, midtones or highlights, do manual curve adjustment, color replacement, channel inversion, reordering, and hue shifting. Doesn't that sound fun? For our purposes, however, we are just going to animate the Add parameter under the Master Controls tab, meaning we will add color to the entire RGB range. By the way, this is easier in the Tremor-style than in Shake style because of the wide layout.

Make sure you are looking at the Master Controls sub-tab, hold down the V key and drag to the right along the Add row so the numbers go to around .3. I add a little bit more blue by holding the B key and dragging some more.

before after - oops!

You will see of course that you have boosted up not only the hill but the sky as well. This is because we are color-correcting a premultiplied element coming out of a CG render. When we use the Over operation mode in a Layer node, we are declaring that the black areas in the RGB channels match the black areas in the mask channel. When we modify the Add parameter, we have boosted the black levels up, so they don't match the black areas of the alpha channel. That is what happens when you color correct a premultiplied element. We are using the ColorCorrect node because even though there are eight billion parameters, you can easily correct this premultiplication problem with a button click or two.

Jump to the Misc tab and specify that you are working on a premultiplied element:

premultiplied - off premultiplied - on

We have now only color-corrected the hill element and not the sky behind it. Life is good.

Ok, we want to animate this now so it will darken up when the cloud passes over it.

Jump to frame 31 when the cloud starts to pass overhead. We go a couple of frames earlier to give a little anticipation to the animation.

Next to every parameter is a mini autokey button . When you press it, you have entered a key. When it is on and you change a value, you have also entered a key. Therefore, you have just entered a key at frame 31.

Jump to frame 37 and drop your add values back down to 0, with maybe a little red in it

frame 31 frame 37

 

Color Correction using other nodes

To talk about color corrections without using the ColorCorrect node, we want to boost up the black levels on the smoke element, so instead of having black and white, we have maybe a grey and white.

There are two principles about color correction. The first is that many of the Color nodes concatenate together when placed one after ther other. Concatenation gives you two primary benefits of greater color integrity and faster processing. What the hell does concatenation mean? Have a good time looking up that word in your English - Nepalese translation dictionary. Basically, it means that the multiple mathematical equations are collapsed into one operation. For example, if you apply a Brightness operation, which multiplies color, with a value of 5, and then apply a second one afterwards of .2, they cancel each other out since RGB * 5 * .2 = RGB * 1. If you did this without concatenation, you would end up with most of your image crunched down to a 20% value. The nodes that concatenate all have a tiny yellow C on the node icon. The nodes with a red C (AdjustHSV and LookupHSV) only concatenate with each other:

So for example, with these six nodes, you could line up Brightness, Clamp, Compress and ContrastRGB in any order as many times as you want and they would all resolve into one operation. AdjustHSV could only be lined up with itself or a LookupHSV.

Note that when working in float space, the concatenation restrictions disappear, since all notes effectively concatenate.

The second thing we need to think about is if you are working on a premultiplied element or not. I'll explain this as I go.

To raise your black levels up, a good node to use is Color - Compress, since you can boost the blacks but leave the whites where they are. Select the SetDOD node coming off of the smoke node and insert a Compress node. By placing it after SetDOD2, it will concatenate with Fade1. If we placed Compress1 between smoke and SetDOD2, the SetDOD would break the concatenation between Compress1 and Fade1.

Boost your Low Color up to perhaps .5 by holding down the V key and dragging. You might also want to add a little blue, what the heck, go nuts.

Low Color of 0,0,0 Low Color of .5, .5, .5 - oops.

Here, you can see a problem similar to the one we had with the hill - we are boosting up all of our black levels on the smoke element, not just the ones where the smoke is. We can correct for this by multiplying Fade1 by the alpha mask, basically restoring the relationship between the black areas in the alpha mask with the black areas in the RGB. The node to do this Color - MMult. MMult stands for Mask Multiply. However, when we stop to consider this, we remember that the smoke came out of a CG render, and is therefore already premultiplied. The term premultiplied suggests to us that it has already been multiplied by its alpha channel. We've just done this again with an MMult node, which means that the image has been multiplied two times by its alpha channel. This runs the risk of degrading our carefully antialiased edges. We therefore need to mathematically compensate for this by first dividing by the mask, doing our color correction, and then multiplying it back by the mask. This process is called unpremultiplying, and it is done to premultiplied images when you want to color correct them. The node to do this is called Color - MDiv, or Mask Divide. Insert it before the Compress1:

Without MDiv, MMult With MDiv, MMult

This unpremultiplying process is handled by ColorCorrect when you specify that the image is premulitplied. You therefore have two choices when color correcting - use the ColorCorrect node and toggle on the premultiplied flag, or use the MDiv/MMult pair to surround your color correction nodes. You should also always be on the lookout for opportunities to group your color correct nodes together to take advantage of concatenation.

For more information, jump to Overview - Premultiplication and Overview - Color Corrections.

 

Applying Vanilla's Shadow

The last step is to discuss three different strategies for working the shadow element onto the mid plate. We will start with the most simple.

The shadows element is a single channel black-and-white image. There is no alpha channel. This means if we subtract the shadows element from the mid element, it will affect the RGB channels of mid but not the alpha. This is a good thing.

Select the mid node and attach still yet another Layer node. Rename it ShadowSubtract or some other clever useful name. Plug the SetDOD from shadows into the second input:

Switch the operation of ShadowSubtract to ISub. This means "Image Subtract". Aren't we clever at Nothing Real? You will notice IAdd, IMult, IDiv, ISub and ISubA. The last one takes the absolute value of the subtraction, making it a good node for finding the difference between two images.

When you switch the operation to ISub, a percentage slider appears. Since the shadow is absolutely black, slide the percentage back to maybe 20 or 30 percent:

So the first possibility of applying the shadow is by using layering math.

Save the script by hitting Ctrl+Shift+S to Save As, and name it vanilla2.shk or something artistic like that.

The next possible way of applying the shadow is instead as mask to a color correction. We are going to replace the ShadowSubtract node with a Brightness node, which can easily be done with a hotkey. Select ShadowSubtract in the Node View, and then Ctrl+click on Color - Brightness. We then plug the output of SetDOD1 (the one attached to shadows) and plug it into the side input of Brightness1. This is the mask input that can be applied to any node:

If you drag the value slider in Brightness down to perhaps .3, you will chagrined and no doubt humiliated to discover absolutely nothing happens. This is because the mask takes the alpha channel of the mask input (SetDOD1 in this example) as the masking channel. If we recall, shadows only has a single black-and-white channel. Therefore, the mask is considered to be black, which means nothing happens. To change this, load the parameters of Brightness1 up. Among the first is a parameter called Mask. If something is attached, there will be a subtree plus sign . Open it up and you will see a set of parameters pertaining to the mask. The one we want to change is maskChannel. By default, it is set to A, for alpha. Switch it over to R, G, or B. Since SetDOD1 is a black-and-white channel, any of them will work:

You shadow is now applied this time with the shadow as a mask to a color correction.

You should use the side mask input in the following circumstances:

If you use the same mask several times on connected nodes, you will:

As an example, let us suppose you wanted to also blur the shadow area (I don't know, I just picked a node at random). You would therefore pick the Brightness1 node and attach a Filter - Blur, and then plug in the SetDOD1 as a the mask input. Don't forget to open the Mask subtree and specify the proper channel:

Don't do this. This is naughty

What this does is mix the brightened area with the non-brightened area, and mixes the blurred area with the color corrected node. This means you are doing two mixing operations. Although Blur and Brightness don't concatenate, you'd definitely be breaking it if they did.

The alternative is to instead use the Layer - KeyMix node. This mixes two images through a third mask. Select the mid node, and then Shift+click on KeyMix. This will shoot off a new branch instead of inserting KeyMix1 between mid and Brightness1. Hook Blur1 into the second input of KeyMix and plug SetDOD1 into the third input of KeyMix1 - not the side mask input. To disconnect the noodles into the mask inputs of Blur1 and Brightness1, either place the cursor over the noodles so they turn red and hit the Delete key, or select the Mask textfields of both nodes and clear them of the word SetDOD1:

Instead do this. This will bring glory and fame

Just like the mask inputs, you will have to specify in KeyMix the channel of the third input you want to use as a mask. You can also invert the channel if you want:

You have now done the shadow with a third technique. You can pile as many operations as you want on the Brightness1 - Blur1 chain and have them efficiently mixed into just the shadow area.

To wrap this up, the two main compositing nodes are Over and KeyMix. For Over, you can either use the Over node or Layer set to Over in the operation.

Different Ways to Insert Nodes into a Tree

To insert a node between two nodes, pick the parent node and click on the new node from the function tabs:

Select the parent ...and click on the new node

To create a new branch, select the parent, and Shift+click on the new node:

Select the parent ...and Shift+click on the new node

To replace a node, select the node to replace, and Ctrl+click on the new node:

Select the node to replace ...and Ctrl+click on the new node

To create a floating node, Ctrl+Shift+click on the node you want:

Nothing needs to be selected. ...and Shift+Ctrl+click on the new node

To extract a node from a tree, select it and hit E for Extract:

Select the node to extract... ...and hit E to extract it.

To delete nodes, select the nodes and hit the Delete key:

Select the node to delete... ...and hit Delete