About Premultiplication and Compositing |
This section is perhaps one of the most important of the Overview, because it talks about compositing and color correcting on a fundamental level. It details the process that makes standard compositing operators work, and defines the term premultiplication (and why you should know about it and order it now for the entire family before the Christmas rush begins), regardless of what compositing software you use.
You should think about premultiplication when you have modify a foreground element, and composite it over a background image. By "modify," we classically mean color correction, and especially nodes that raise the black level such as Add, Clamp, ColorMatch, ColorCorrect, Compress, and Contrast, but there are consequences with filters as well.
Some software packages "take care of it" for you, meaning they hide the state of premultiplication. This works great nine times out of ten, but for that last 10 percent of the time, you have no practical solution. Other compositing packages pretend it doesn't exist and encourage you to chew on your matte, or modify foreground/background Alpha multiplication curves. Taking a third approach, Shake gives you complete and explicit control over premultiplication. Although this does mean it can be painful, it helps you get around problems you typically run into with other packages.
For those of you who don't feel like reading a long-winded dissertation on the mathematics of premultiplication, here is the abridged version:
|
|
Egghead Definition
of Premultiplied
Let's start with the definition of what a premultiplied image is:
"An image that has its RGB channels multiplied by its Alpha channel."
Typically, images from a 3D render are premultiplied. This means the transparent areas have black in both the RGB areas and in the same areas of the Alpha channel. A side effect of a premultiplied image is that the RGB channels never have a higher value than the Alpha channel. The beauty of all this is that it has absolutely no meaning without understanding how compositing works, which is why this section is very long. So, jam those toothpicks between the eyelids and read on.
Problems Caused
by Ignoring Premultiplication
Aside from being ostracized by your friends and perhaps detecting a faint musty smell (just kidding here this is a commonly misunderestimated concept), there are two typical problems that occur when the premultiplied state of an image is ignored.
The following example demonstrates these problems:
The second image, munch_pre.sgi, is rendered out of Maya, is premultiplied (the blacks in the Alpha match the blacks in the RGB) and has an embedded Alpha channel. This image, by the way, is of Munch. (We aren't sure what Munch is. He's got this cool conical tongue, though, and can shoot lightning bolts out of his head, so that should count for something).
munch_pre.sgi | munch_pre.sgi, alpha channel | bg.jpg |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
If premultiplication status is ignored, you can have problems with edges, or with global levels being raised up. Many people see these types of errors, and assume it is a mask problem, so they pinch in the mask a bit. Even more extreme people have erroneously assumed you can't color correct premultiplied images. Happily for our competitive benefit, these problems are easily solved through proper management of premultiplication (which, still hasn't been explained).
The Math of Over and KeyMix
To understand why premultiplication problems occur, it is best to cut straight to the heart of compositing by understanding how a standard composite operator (foreground through a mask over a background) works. This has nothing to do with Shake, but in fact was worked out in the 1970s, mainly by two clever (but no doubt fun) fellows named Porter and Duff.
Here is the math they came up with. In this equation, A is the foreground's Alpha channel:
Comp = (Fg * A) + ((1-A)*Bg)
Rather than go into this too deeply, look briefly at the signficant part, Fg*A. This is the definition of premultiplication remember "RGB multiplied by its Alpha "? Since you don't want to sweat about the math, you can alternatively understand this by actually building the composite with other nodes, in effect reconstructing an Over node from scratch.
In the following example (continued from the section "Problems Caused by Ignoring Premultiplication"), build the compositing equation:
munch_unpremult.jpg | munch_mask.iff |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Tree | IMult2 |
![]() |
![]() |
Comp = (Fg * A) + ((1-A)*Bg)and you end up with exactly the same result as an Over node:
![]() |
![]() |
By punching a hole in the background, the Alpha determines what is transparent when the two plates are added together. The key concept here is that because you are adding, anything that is black (a value of 0) does not show up in the composite.
IMult1 | IMult2 |
![]() |
![]() |
Since the KeyMix and Over nodes do all of this math for you, you don't have to create four nodes to do a composite. What is the difference between Over and KeyMix? Over assumes that the foreground is premultiplied, (identical to IMult1). KeyMix is only for non-premultiplied images (identical to munch_unpremult). Therefore, strictly speaking, the math breaks down like this:
KeyMix = (Fg * A) + ((1-A)*Bg)Over = Fg + ((1-FgA)*Bg)
In these formulas, the foreground of Over is already
multiplied by the foreground Alpha channel, thus the term premultiplied
it isn't multiplied in the composite because it was previously multiplied,
usually by the 3D renderer.
Unpremultiplying an Image
With this knowledge, you can go back and start to understand the errors that occur when the ContrastLum and Add are used with the Over.
![]() |
![]() |
The clue is to look at Add2. There is no longer an exact correlation between the black areas of munch_mask and the black areas of the RGB channels:
Add2 | munch_mask |
![]() |
![]() |
Tree with IMult3 inserted | IAdd2 |
![]() |
![]() |
Although this appears to work, mathematically speaking, there is an error. You need to have this:Comp = (Fg * A) + ((1-A)*Bg)but in fact, the above example multiplies the foreground twice (there are two IMults), so you have this:Comp = (Fg * A * A) + ((1-A)*Bg)
Add switched to ContrastLum | IAdd1 detail |
![]() |
![]() |
If IMult3 is ignored (to ignore a node, select it and press I), you end up with the same nasty edges as before, so don't do that.
IAdd1 detail with IMult3 ignored |
![]() |
So, what are you to do? Returning to your fourth grade math class, if you divide something by a number and then multiply by the same number, it is the same as multiplying by 1 the equivalent of no change at all. Therefore, if you multiply the foreground by the mask one too many times, divide it by the Alpha to balance it out. Mathematically, this looks like this:Comp = (Fg * A / A * A) + ((1-A)*Bg)or, abbreviating it, we return to the proper formula, since the two extra A's cancel out:Comp = (Fg * A) + ((1-A)*Bg)
IDiv inserted | IAdd1 detail |
![]() |
![]() |
By dividing by the mask, the image is un-premultiplied amd ready for color correction. Therefore, you can conclude that color correction should be done on an unpremultiplied image.
IDiv and IMult3 deleted, ContrastLum moved up | IAdd1 detail |
![]() |
![]() |
You can therefore derive this little rule:
|
Managing Premultiplication
The above steps have all been an elaborate illustration to explain Over, KeyMix, and premultiplication. Now the basic difference between KeyMix and Over:
KeyMix
|
Over
|
For non-premultiplied foreground images. The Alpha can be anywhere, including in the foreground image. | For premultiplied foreground images, but can also turn on premultiplication if necessary for unpremultiplied images. The Alpha is in the foreground element. |
Naturally, Shake does not require you to constantly separate your Alpha and to perform IMults and IDivs with the second image. Instead, it has nodes to do this for you. The following examples duplicate the current complex tree with simplified versions.
This version uses KeyMix, which handles un-premultiplied foreground elements, the background, and a mask to split between the two. Activate invert in the KeyMix parameters (you could also switch the foreground and background inputs):
Identical tree using KeyMix |
![]() |
If you are using a premultiplied image straight out of a 3D
render, there are special tools that unpremultiply and premultiply the RGB
channels by the Alpha . These tools are Color - MDiv and Color
- MMult. "MDiv" stands for "Mask Divide," "MMult"
for "Mask Multiply." Used in a node tree, MDiv unpremultiplies
the image, then you color correct, and then premultiply using MMult.
With the premultiplied image as the foreground, the tree looks like this:
Identical tree with a premultiplied Foreground |
![]() |
Over has a premultiplication shortcut. In this tree,
turn on the preMultiply flag to omit the MMult entirely:
Identical tree with premultiplication handled by Over1: | Over1 parameters |
![]() |
![]() |
You don't need to apply an MDiv for each color correction. You can stack
as many correctors up as you want:
Multiple corrections on one MDiv |
![]() |
Filters and Premultiplication
Now to our next step using filters (like Blur). Here is rule number two:
|
In this example, see what happens when an unpremultiplied image is filtered.
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Blur on a premultiplied image | Over1 detail |
![]() |
![]() |
Since Transforms do a bit of filtering, technically speaking you should do Transforms on a premultiplied image as well, although your tolerances are a little looser.
Typically, ensure your image is unpremultiplied by inserting an MDiv onto premultiplied images, do your color correction, appy a MMult, apply transforms and filters, and then composite.
Nodes Useful or Particularly Pesky When Dealing With Premultiplication
The following nodes can generally switch the premultiplication status of an image, although anything that operates on the mask differently from the RGB affects this status.
Node | Details |
ColorCorrect | This color corrector has an option in its Misc tab to indicate if the incoming image is premultiplied. If it is, set it to "yes", MDiv and MMult are internally inserted into the computation. |
Reorder | This allows you to switch a channel into the Alpha channel, and could disrupt your premultiplication status. However, it isn't often used on images in the normal chain of color correct, position, composite, but is more of a utility node. |
DilateErode | This node is typically used to add or chew the matte by a few pixels. Set your channels to just "a". Since you then modify the matte separately from the RGB, you make the image unpremultiplied. |
LumaKey, DepthKey, DepthSlice | These keyers all have the option to immediately set your image to premultiplied. |
KeyLight | This keyer can output either a premultiplied or unpremultiplied version, or just the Alpha channel with the RGB untouched. |
Primatte | This keyer can output either just the Alpha or a premultiplied version. Setting it to unpremultiplied requires an external MDiv. |
AddMix | This modified Over node allows you to manually break the premultiplied relationship by tweaking curves of how the matte multiplies the foreground and background images. It was inherited it from another package, where it was used extensively because this particular system had no control over premultiplication. |
SwitchMatte | This copies a channel from the second image to the Alpha channel of the first image. You have the option to immediately premultiply the image (enabled by default). |
DropShadow | This should be applied to premultiplied images only. |
|