Tutorial 3.5: Tracking and Stabilizing |
This tutorial discusses the three tracking nodes in Shake: Tracker, Stabilize, and MatchMove.
The first section ("Eat At Joe's") corrects a clip with a drifting camera. It discusses the basic mechanics of tracking, and how to convert stabilization data to matchmove data.
The second example ("Get On the Bus") discusses tracking guidelines and executes a classic 4-point track to match a sign onto a bus side. As original as eggs for breakfast.
For more detailed information on tracking parameters and how a tracker functions, jump to About Tracking. For formatting examples of the three tracking nodes, jump to the Tracker, Stabilize, or MatchMove reference pages.
Tracking allows you to do two things: First, you can eliminate unwanted movement in a plate, for example, the movement caused by film gate weave. This is known as stabilizing. Second, you can match a stable element to a moving element, such as swapping the heads of two actors. This is called matchmoving. Both involve tracking patterns in an image sequence from frame to frame. This can be done manually, but tends to get really tired, really fast. In Shake, three nodes track sequences. The three nodes are:
In the first section, stabilize a moving clip, and then switch the
stabilization data to track an image to the moving clip in a matchmove.
![]() |
![]() |
Add an Image - FileIn node and read in doc/pix/stabilize/stab.@.jpg, a 15-frame clip.
In the FileIn node's (stab) Global parameters, click Auto on the timeRange parameter.
Click Home on the Time Bar.
This does not affect your rendered frames, but helps you adjust your track points.
You have by now no doubt discovered the plate drifts to the left. This is a prime candidate for stabilization, which locks the image down.
A tracker appears in the Viewer.
When the cursor is positioned over the tracker box, the tracker is highlighted. You can move a highlighted tracker box in the Viewer.
The tracker is composed of the track point, the reference pattern,
and the search region. A pattern is isolated with the inner box called the reference
pattern. Once tracking begins, the tracker looks for that pattern within
the larger box called the search region. When a match is found,
a track point (keyframe) is created.
The 15 frames are tracked.
![]() |
When the plate drifts, the Stabilize node moves it back to the original point. This eventually causes a black border appear around the image. The simple (lazy) way to fix this is with a Scale node to adjust the plate size. This concatenates with the Stabilize node. This is good.
Connect a Transform - Scale node to Stabilize1.
Scale the image up slightly. Examine frame 15 to gauge the
correct scale.
As this is the Lazy Man's Method, and therefore appealing to compositors across the world, it is not the ideal method. Instead, a clean plate should be constructed and placed behind the image. See How to Make a Clean Plate.
Use the same building clip to build a simple matchmove. In this case, remove the stabilization data from the building clip, invert the stabilization data, and apply the data to a second image.
Add another FileIn node (don't delete your process tree from the previous section) and read in doc/pix/stabilize/eatatjoes.iff.
Since you have already tracked the background, you can attach
the Stabilize1 node to the EatAtJoes node, and then use a Layer
- Over node to composite the two together.
Extract the Stabilize node (select the node in the Node View and press E) from the stab (building) node.
Connect the extracted Stabilize node to the stab2 (EatAtJoes) image.
Add a Layer - Over node, and connect the stab (building)
node to the second input (background) of the Over node.
The EatAtJoes image drifts away from the background image. The Stabilize node is trying to stabilize the imagethe opposite of what you want. To match the EatAtJoes image to the movement of building clip, invert the transform with the shrewdly-named invertTransform button (by setting it to match).
The EatAtJoes sign moves with the background plate.
As shown, you can use Stabilize for stabilization and matchmoving. Others lurking around the office prefer to use the MatchMove node, described in the following section. There is one gotcha with using Stabilize for matchmoving: You must turn off the transform and connect the Stabilize node to the background (tracked clip) in order to re-track.
![]() |
The next example uses more complicated tracking information to match a background clip that changes perspective. The MatchMove node is used to match the movement of the background. That's where the devilishly clever name MatchMove comes from. In this example, read in a clip of a moving bus and an advertisement image, and apply the image to the side of the moving bus. This tutorial also discusses some solutions for common tracking problems you are likely to have in future projects.
Select New Script from the File menu or press Command+N
/ Ctrl+N to clear the existing nodes and begin a new script.
Add an Image - FileIn node and read in the 39-frame clip
bus2 (nope, there is no bus1) from the doc/pix/bus directory,
and read in sign.jpg from the doc/pix/ directory. The
sign image is 720x210 pixels, and the bus clip is 720x486 pixels.
The office of the Shake team appears in the clip. The building, not the bus.
The following illustration shows is what you are trying to achieve in this
lesson. Yeah, this is the shot you put first on your reel, right after "I
am a dork. Don't hire me."
Before | After |
![]() |
![]() |
The first step is always to examine the clip and identify good search patterns. Luckily, the sign of the bus is visible during the entire clip, although there is some distortion due to perspective shifts.
In the Node View, select the sign node and add a Transform - MatchMove node. Unlike other Transform nodes, MatchMove allows you to composite within the node. The tracking is done on the second input (the background).
Connect the bus2 node to the second input of the MatchMove
node.
Although this project calls for a 4-point track (one for each corner), begin with only one tracker to keep things simple. Several intentional mistakes occur in this tutorial to help you better recognize problems with tracking your own footage. As they always do.
Go to frame 1.
Position the tracker in the lower left corner of the sign on the side
of the bus. Leave the tracker at the default size, and precisely position
the tracker on the corner. Use the + / - keys to
zoom in, with the cursor as an aiming device. Press Home to return
to the normal size.
Doh! The tracker advances a few frames and stops. The last number in the text next to the tracker box, reads "c=.2" (approximately). This indicates the correlation, or accuracy of the track. A score of 1 means a perfect correlation with the reference pattern between frame 2 and frame 1 (frame 1 as the reference pattern). A score of 0 is amazingly bad. On frame 2, you have a correlation of .2. This is bad.
The search region is too small to encompass the movement in the
clipthe bus moves too far to the right. In this illustration, the desired
area, artificially marked in green, extends beyond the zone of the red search
region.
If the search region is too large, you waste time processing.
If the search region is too small, the tracker does not find successful matches.
There is also a black line around the sign along the bottom of the pattern.
There is a chance due to the perspective shift in later frames that the tracker
will confuse the two lines. Adjust the search area with these in mind.
Go to frame 1.
Grab the corners of the search region and scale it outward, but shrink
the bottom line up a bit. Also scale down the reference pattern to better
fit the corner.
![]() |
Click track forward.
Gee, everything was going fine until around frame 34, right?
At that frame (results vary), the tracker spitefully decided that the words
on the sign were the best match to the original reference pattern at frame
1.
The tracker continually compares the new images back to frame
1 (or whatever frame you start the track). As the bus moves away, the sign
gets smaller. At a certain point, the words match the original thickness of
the sign border from frame 1 better than the sign border itself.
This matching behavior is controlled by the referenceBehavior
setting in the tracker parameters under the tolerance behavior subtree.
It is set to use start frame by defaultit compares the new samples
to the original start frame (not necessarily frame 1 if you start at a later
frame).
Two settings in the referenceBehavior can help with scaling changes in the image sequence. The settings are update every frame, that uses the sample of the previous frame for the reference pattern, and update if below reference tolerance, that checks the correlation of every frame. If the correlation falls below the referenceTolerance value (right above referenceBehavior), it uses the previous frame as the reference pattern. It continues to use that pattern until the correlation once again dips below the referenceTolerance. The second one (update if below...) is more accurate because you get inherent drift with update every frame as all of the tiny errors accumulate.
Although you can track from frame 33 (the frame before the tracker planted), it is better to find a frame with a high correlation. The correlation is displayed in the yellow text next to the tracker box. Ideally, this is of course frame 1, but you may find a high correlation at approximately frame 14.
Find a frame with a high correlation (above .97).
You should get a good track with these settings.
You can also manually enter track positions with the use of the Viewer AutoKey.
Well, that's one track. Now the other three trackers.
The three extra trackers, pre-positioned on the image, are activated. Each tracker is located in the relative corner where it should stay. For example, the upper-left corner of the foreground image is connected to track4 of the MatchMove. Shake uses Cartesian coordinates, so the trackers are ordered according to the following image.
Yes, you are right, this is different from some other prominent systems. Deal with it.
The first track point, track1, is already set and its keyframes are
visible.
Every time you start a track, all visible trackers analyze the image and create new keyframes. Therefore, turn off the visibility of track1. The four trackers are listed in the lower portion of the MatchMove parameters. Click the track name text field (that is, where is says "track1") to change the name. Change the display color of a tracker by clicking on the color picker button. The V button toggles the visibility of a tracker.
An active tracker is green in the tracker list, and yellow in the Viewer.
Go to frame 1.
Following the same principles of the first tracker, position and shape
each of the remaining trackers on the remaining sign corners.
![]() ![]() |
With the four tracks plugged into the MatchMove, you can now apply the foreground element.
In the MatchMove parameters, the outputType list contains several compositing options. Background is selected by default to allow you to immediately start tracking. To test the track, switch the outputType to a compositing operation.
This produces a somewhat less-than-convincing composite.
As with Stabilize, you must activate the transformation.
![]() |
![]() |
If an image appears brighter in an Over operation (in effect adding the two images), you should immediately suspect the absence of a mask in the foreground. Since the sign image is a JPEG and does not support alpha channels, you must add an alpha channel to the image.
Open the sign's FileIn parameters.
Activate autoAlpha.
![]() |
![]() |
This is an improvement. The four corners of the foreground image are snapped to the four corners of the trackers. If the image is twisted around, you have done something such as put track3 in track4's corner. You can unravel that in the next step.
The sign probably may not fit exactly into the background's sign limits.
A little bit peeks out from behind. You therefore need to adjust the position
of the foreground sign.
Go to frame 1.
Ensure the Viewer AutoKey button is deactivated (unless you want to animate it, which you don't).
Click the BG/FG toggle button
to show the foreground display
. This displays four corner markers in the Viewer. These points correspond
with what is matched up to the four trackers. By default they are located
in the four corners of the image.
The corners represent the track location points. Therefore, bringing the
points in expands the image size in the composition.
Note: You may need to switch your outputType back to Over. Occasionally this parameter is not reset properly.
![]() |
![]() |
![]() ![]() |
Anybody who has visited Los Angeles knows that a clean bus is
a mythical beast. As the final step, color correct the sign to better match
it to the bus.
Before | After |
![]() |
![]() |
In the Node View, select the sign node.
Note: You do not need a MDiv/MMult set because
you are working on the full frame.
For the color correction, first adjust the whites and blacks with the Compress node. Because the sign has pure whites and blacks, and the bus has good representations of what should be white and black, Compress works well.
In the Compress1 parameters, click the Low Color Picker box.
Scrub color in the Viewer that represents the blackest part of the bus, for example, under the wheel well of the bus.
The blacks are raised to around 18 percent.
Click the High Color Picker box, and select a bright color from the side of the bus.
As the bus changes its lighting levels as it turns, animate the High Color parameter.
Go to frame 1.
In the Compress1 parameters, click the Autokey button in the High Color parameters.
Go to frame 39.
Scrub the same white area. Since Autokey is already activated, a keyframe
is created.
The sign is a neutral grey color. Some warm color makes it stick out a bit and draws the viewer's attention (though not too muchit still has to be integrated into the scene). You can tint it yellow with Compress1, but you then must have the same modification for all of your keyframes, which is cumbersome. A better idea is to use the Mult node to tint it. As Compress and Mult concatenate, there is no loss of color quality. This is good. Very good.
Load the Mult1 parameters.
Using the color picker, tint the sign yellow.
The sign is now yellowish, but the red of "Joe's" is too bright. Use the Saturation node to lower the overall saturation of the sign. You may need to boost the Mult1 node to compensate.
Note: Saturation does not concatenate with other nodes.
The three nodes (Compress1, Mult1, and Saturation1) demonstrate an advantage of working with nodes and concatenation. Because Compress1 is animated, it is awkward to apply adjustments to the overall color for the entire clip length within the Compress node. By breaking these steps into separate nodes, you don't have to adjust each keyframe.
Next, because the Compress raised the black levels in infinite space, the Color - SetBGColor is used. This sets area outside of the DOD to a specific color (black by default). You can also use a MMult since the mask is pure white, but you don't want any confusion about the premultiplied status. If the sign has a soft mask, you must use a Mdiv/MMult pair.
To test the effect of the SetBGColor1 node, select the node and press I to ignore it.
Press I again to activate the SetBGColor1 node.
The sign is color balanced in the scene, but the shadow, created with the AddShadow node, needs to be adjusted. Without this node, the sign has no feeling of depth.
In the AddShadow1 parameters, set the xPixels and yPixels to approximately -5.
Set fuzziness to approximately 35.
Set shadowOpacity to approximately .7.
No Shadow | With AddShadow |
![]() |
![]() |
![]() |
You may have picked up that tuning the corners of the sign using the FG/BG Viewer controls was less than ideal. Some might use the word "maddening and suicide-inducing," as there is no interactive feedback. An added drawback is onscreen controls from upstream are not properly adjusted by the MatchMove node. This is one advantage that the Stabilize node has over the MatchMove node. As demonstrated in the Eat At Joe's tutorial, you can use Stabilize instead of MatchMove to create matchmoves. This gives you a more intuitive control mechanism, but requires more plumbing in the Node View. Note for this exercise, you are copying the tracks from MatchMove1, but you could have generated them with Stabilize.
The following is the general workflow for matchmoving with Stabilize:
This technique has several advantages:
Switch the tracking from the MatchMove node to a Stabilize node:
In the Node View, select AddShadow1.
Shift-click Transform - CornerPin.
Connect a Transform - Stabilize node to the CornerPin. Normally, you attach the Stabilize to the bus2 node to start tracking, but you already have your tracks in MatchMove1.
Connect an Over node to Stabilize1.
Connect bus2 to the second input of Over1.
Next, copy the tracks from MatchMove1 to Stabilize1.
![]() |
![]() |
A pop-up window appears that lists all tracks.
Select MatchMove1.track1 and click OK.
Repeat the process for tracks 2 through 4.
Activate the transformation with applyTransform.
Set inverseTransform to match:
Things don't quite line up in the Viewer. Oops.
Frame 1 | Frame 30 |
![]() |
![]() |
MatchMove has controls for adjusting the foreground position. Stabilize, by default, assumes that frame 1 has the relative position that you want. The selected frame can be modified with the referenceFrame parameter, but it still doesn't modify the relative position. You must therefore rely on external nodes to adjust this, in this case the CornerPin node.
Ensure the Viewer Autokey button is deactivated.
Load the CornerPin1 parameters.
Move the corners of the CornerPin to match the side of the bus.
The controls are now completely intuitive, as well as interactive. Just like the DMV.