ImageMagick Examples --
Masks
- Index
- ImageMagick Examples Preface and Index
- Alpha (Matte) Channel
- Using Masks with Images
- Special Image Masks
- Regions and Region Sub-Images
- Background Removal
- Hole_Filling (Under Construction)
Alpha Channel
The transparency (alpha) channel of an image is completely optional, and often requires special handling separate to the normal 'color' channels. See Image Color Space above. The existence of a transparency channel can also effect how the various operators treat the other color channels, generally because a fully-transparent color should often be completely ignored by an operation. If this was not the case you get 'Black Halos' around images, such as was seen in major IM Bugs in the early days of IM v6. For example the Resize Halo Bug, and the Blur with Transparency Bug. To make matters worse, this channel is also sometimes referred to at an image's 'transparency' or 'opacity' channel, or even the image's 'mask'. All however refer to the same, special, fourth channel of the image. To explain the difference we need a working example image and for this I'll use a PNG image of a 'crescent moon' image (from a CopyOpacity Composition example). Now as you can see this image has a lot of areas which are fully transparent. Not only that I needed to save the image using the 'PNG' image format which is one of the small number of image formats that properly understands and handles transparent and semi-transparent colors.
I can demonstrate this transparency by overlaying the image onto the IM built-in checkerboard pattern, using Alpha Composition.
|
Internal Alpha Channel
Now internally IM v7 stores the transparency information in a 'alpha' channel, which just like the color channel is just a plain grey scale image of values which range from white, for fully-transparent (or clear), to black for fully-opaque. It is sort of like what you would get if you look at a silhouette of the original image. Low level operators such as "-level
" and "-threshold
" handle the data as alpha. Check the Official Option Reference if you are unsure.
Here is a very old way to extract a 'alpha' transparency values from an image. It saves the transparency channel as a 'alpha' image file format, and required two separate steps, and commands to define the right image file format.
Controlling Image TransparencyThere are two operators that give you low-level control of the transparency channel of an image in memory. The newer operator "-alpha " methods are now the recommended method of control, though many IM Examples still show and use the older "-alpha " operator. An image cannot only have alpha channel data, but it also has a 'switch' that defines if the channel data is viewable or valid. This means images can have three states with regards to the alpha channel.
-compose CopyOpacity -composite " will always turn on the alpha channel in the resulting image, as it is the operator's job to copy data into an alpha channel. As such it must exist in the final result. However its existence in the input data can have other consequences. See Copy_Opacity Composition Method for more details. Similarly creating a canvas using the color 'None ' will also automatically create and enable the transparency channel, so as to ensure the blank image really is transparent. On the other hand, creating a canvas using some other Color Name will generally not create any transparency channel as images are opaque by default.Here are the various " -alpha " methods and examples of how they effect images and their transparency.
Alpha Off or "This is just a simple switch on the image, which turns off any effect the transparency has on the image. It does not actually delete or destroy the alpha channel attached to the image, it just turns off any effect that channel has on the image. Similarly no operator will effect the attached alpha channel while it has been turned off. For example let's use the 'crescent moon' image (from a CopyOpacity Composition example), and simply turn the image alpha channel off.
Note that the moon shape completely vanished when the transparency was turned off, though that is actually rarely the case. Basically even the 'transparent' areas have color, which is just not normally visible, in this case the hidden color was the fractal canvas image that was used to create the moon image. This hidden color could be anything, from a simple GIF Transparency Color, that the GIF format uses to represent transparency in its color table, to garbage colors left behind during the images creation, as above. More typically the transparency color is simply pure-black for any pixel that was fully-transparent. Note that pixels close to the edge may be semi-transparent, and thus still have a valid color that is only partially visible. The " |
|
|
Alpha On
The "-alpha On
" is the exact opposite to the previously looked at Alpha Off method. Typically this is too simplistic for the purpose you are wanting and as such should be very RARELY used. You should use "-alpha Set
" in almost all cases. Basically 'On
' method just flips the switch so that the image transparency data is visible again. Any existing transparency data is not modified, so if the in-memory image still has some old alpha channel data, that data will suddenly be visible again. For example, here we turn 'Off
' the transparency data, then immediately turn it back 'On
, reproducing the original image.
|
-shade
". For an example of this special usage see Shaded Shape Images.
Alpha Activate/Deactivate
enables and disables the alpha channel, respectively, with persistence. This is like on/off in Imagemagick 6. In Imagemagick 7, -alpha off will remove the alpha channel permanently such that -alpha on will not re-enable it.
Alpha Discrete
treat the alpha channel independently (do not blend).Alpha Opaque
This method not only ensures the alpha channel is 'active' but that it is also completely opaque, regardless of if the image had transparency 'activated/on' or 'deactivated/off'. For example...
|
-alpha off
" to turn off the alpha channel, then using "-alpha on
" to turn it on, while resetting it to be opaque.
|
-alpha off -alpha set
", though you may as well use "-alpha opaque
" in that case.
Alpha Transparent
Similarly this ensures the alpha channel is 'active' but also fully transparent.
|
|
Alpha Extract
The 'Extract
' method will simply copy the 'alpha' mask of the image as a gray-scale channel mask.
Note that fully-opaque is white, while fully-transparent is pure black. As image contained some semi-transparent pixels along the edges (for anti-aliasing providing the images shape with a smoother look), this image is not pure black and white, but also contains some gray colored pixels around the edges. If your ImageMagick is an old IMv7 version, this is a (near) equivalent technique, using channel extraction.
magick moon.png -channel a -separate +channel -negate alpha_extract.png |
Extract
' method will also turn 'Off
' the alpha, but it is not cleared, so turning the alpha channel back 'On
' will re-create a shape mask of the original image.
|
|
Alpha Copy
The 'Copy
' method is the reverse of 'Extract
', and essentially performs a CopyOpacity against itself. That is, it will turn a gray-scale image (regardless if its alpha channel is enabled or not) into a shape mask image.
It does not matter if the image had an existing alpha channel or not, all it does is create the images transparency from the image gray-scale values. Once you have a shape mask, you can use various Color Tinting or Duff-Porter alpha composition methods, to color it. For examples of using a shape mask see Masks as Colored Shapes.
Alpha Shape
To make use of a gray-scale image easier, the 'Shape
' method not only creates a shape mask (as per Alpha Extract, but will also color it using the current background color.
This means you can very quickly color a gray-scale mask simply by shaping the image, then flattening it onto a different background color
|
Background is not actually the right color to use for this 'shape' coloring operation. It should be using the 'fill' color to set the shapes foreground color. As such which color should be used is likely to change. Background is only used due to internal difficulties in accessing the current fill color. This change will likely happen as part of IMv7. |
|
The above will map the colors using a linear colorspace, and may need to be converted to sRGB at some point to get a more visually correct gradient of colors. |
Alpha Remove
The "-alpha Remove
" method (added to IMv7.7.5) is designed to remove the transparency from an image, using the current "-background
".
|
Alpha Background
As of IM v6.5.2-10, a 'Background
' method was made available that will set the hidden color of fully-transparent pixels to the current background color. Normally this color is of no consequence, as it can only be seen if the alpha channel is turned off. However the color of fully-transparent pixels is saved in PNG Image file format, and for large images, having random unknown fully-transparent colors can significantly effect its compression handling. See PNG with Better Compression and the IM Forum Discussion Eliminating alpha channel garbage for more details. Note that no color mixing is applied, only a direct color assignment to any fully-transparent color. The pixels however will still remain fully-transparent, and as such you will see not change to the image.
For example, here I use it to set all fully-transparent pixels to 'HotPink '.
|
As you can see this made no change to the actual look of the image. To see the change we will now turn off the alpha channel.
|
This is not the same as Removing Transparency
The edges of the shape will have made all semi-transparent pixels opaque, and as a result produced some strong aliasing (stair-cased) edge effects. Note that even the normally opaque only format PNG24, can still save boolean transparency if all the fully transparent colors are the same. For details see the example in PNG Sub-Formats. This process of replacing the colors is actually almost the same as doing a "-channel RGB -fill color -opaque None +channel
". See Direct Color Replacement. Note that many other image processing operators will also magick any fully-transparent pixels, to fully-transparent black (color 'None
'), as this is the color equivalent of a mathematical zero. Here is a summary of some image operations that are known to do this, though none are as direct or as fast as using this operator.
magick moon.png \( +clone -alpha off \) \ -compose SrcIn -composite moon_black.png magick moon.png -channel RGBA -blur 1x.000000001 moon_black.png magick moon.png -channel RGBA -gaussian 1x0 moon_black.png magick moon.png -fuzz 0% -transparent none moon_black.png |
None
'), but also all near-fully-transparent colors (which otherwise does have a valid but practically invisible color), simply by specifying a fuzz factor. It will produce some data loss, but may improve compression in images with lots of near-fully-transparent colors. Often these nearly total transparent pixels can have very odd or wrong colors, and this method will allow you to remove such odd pixels before they cause other problems.
Removing Transparency from Images
While the Alpha Off will simply flip a switch and turn off the transparency channel. You can also get the same effect if you attempt to save the image into a file format that does not allow the use of transparency. For example by saving to JPEG...
magick -size 70x60 xc:none -font Candice -pointsize 50 \ -fill Black -annotate +10+45 'A' -channel RGBA -blur 0x5 \ -fill white -stroke black -draw "text 5,40 'A'" a.png magick a.png a.jpg |
|
Alternative techniques of removing transparency, is to somehow generate a new 'background' or 'canvas' image and Over Compose your image onto that background so that the transparency is replaced. Preferably while preserving the original image's meta-data, such as profiles, labels, captions and comments that may be present. Methods for generating such a canvas is exampled in Creating Image Canvases of Same Size. Here is one such method...
|
|
mogrify
" or with a sequence of multiple images, basically because the "-flatten
" operator is really designed to merge multiple images into a single image. The other common method that does work with multiple images is to give the image a zero sized Border with the appropriate "-bordercolor
". For example...
|
composite
" command to Tile an image 'under' the original, (using Dst_Over). This compose method ensures the original images meta-data and size is preserved.
|
Many of the above methods, are either effected by, or may destroy any virtual canvas information an image may have, as part of its processing. When the virtual canvas is involved, you may need to look at the details of individual operators more closely. In many cases the virtual canvas effects can be useful to your overall image processing. |
Boolean Alpha Transparency
For some image file formats you don't need to completely remove the alpha channel, but only allow pure on/off or boolean transparency. Index (Palette) image file formats such as GIF and PNG8, are typical of this. Examples are currently looked at in GIF Boolean Transparency, but should eventually move here.Outline or Halo Transparency
Sometimes you will like to add an outline around an image containing transparency. One way is to use EdgeOut Morphology to quickly get all the neighbouring pixels to the original image, color them, and then Under (DstOver) Compose it with the original image.
magick knight.png \( +clone \ -channel A -morphology EdgeOut Diamond +channel \ +level-colors red \ \) -compose DstOver -composite knight_outlined.png |
|
Using Masks with Images
Masking An Image
As shown previously there are a couple of ways to mask an image, so as to make part of the image transparent. And which method you choose depends on the whether your image mask is a grayscale mask, or a shaped mask.Editing an Image Mask
The mask of an image is a really useful thing to have. We can for example erase parts of an image very easily by modify a mask of the original image. Remember the "-draw
" operator cannot draw nothing, and currently has no erase option. Here we create an image, then by extracting and modifying its mask, before restoring it to the original image.
magick -size 100x100 xc:none -stroke black -fill steelblue \ -strokewidth 1 -draw "circle 60,60 35,35" \ -strokewidth 2 -draw "line 10,55 85,10" drawn.png magick drawn.png -alpha extract mask.png magick mask.png -fill black -draw "circle 40,80 60,60" mask_bite.png magick drawn.png mask_bite.png \ -alpha Off -compose CopyOpacity -composite \ drawn_bite.png |
-alpha Off
" operation in the above as it is vital to ensure the grayscale image does not contain unneeded transparent channel. And Presto we took a bite out of the original image. We can also re-add a part of the image we removed. For example, here I re-add part of the 'bite' I removed from the original image, by drawing white area onto the mask. The mask is then again returned to the original image using CopyOpacity Channel Composition.
magick mask_bite.png -fill white \ -draw "circle 50,70 60,60" \ -draw "roundRectangle 78,5 98,25 5,5" \ -alpha off mask_bite2.png magick composite -compose CopyOpacity mask_bite2.png drawn.png drawn_bite2.png |
SteelBlue
' color of the original image. You should not count on this if the image was saved to some other file format or further modified.Here is an alternative method of erasing parts out of an image but rather than extracting and modifying a Grayscale Mask, we instead use a Shape mask as a sort of 'erase' tool using DstOut Composition Method.
magick -size 100x100 xc:none -draw "circle 40,80 60,60" mask_shape.png magick drawn.png mask_shape.png -compose DstOut -composite drawn_bite3.png |
Masks as Colored Shapes
An alternative to just using the mask to add or re-add transparency to an image is to actually combine the mask directly with images in various ways. For example suppose we just want to use a mask as a symbol or shape we want to overlay onto an image in various colors. For this we need a mask, which I'll extract from a special 'symbol' font.
|
|
Note the use of 'PNG' image format for generated shaped image rather than GIF so as to avoid problems with GIF Boolean Transparency. |
|
-background
" color which Combine used to fill in the undefined channels of the new image. An older but more complicated way is to use 'CopyOpacity
' composition method to set an image's transparency to the given mask, then use Uniformly Color Tinting to color the resulting shape. This works and for a long time was the best technique to use, but is no longer recommended.
|
|
|
Mathematical Composition
Rather than overlaying the mask onto some background, you may only be interested in coloring the image with just the white or black parts of the mask itself. This is relatively straight forward, simply by using some Mathematical Alpha Composition Methods to change the color of the mask to match a color, tile or other image. For example the 'Multiply
' compose method will replace the white areas (multiply value of 1) with the overlay image, while leaving the black areas (multiply value of 0), black. The 'Screen
' operator is exactly the same as 'Multiply
' but with the images negated so it effectively replaces the black areas of the image. For example, let's use the larger mask image from above, to overlay a larger image generated with a tile pattern.
magick mask_bite.png -size 100x100 tile:tile_disks.jpg \ -compose Multiply -composite compose_multiply.png magick mask_bite.png -size 100x100 tile:tile_water.jpg \ -compose Screen -composite compose_screen.png |
Multiply
' alpha composition method is especially useful for replacing the background of text images (IE: black text on white background), such as images generated from Postscript Documents.
Masked Alpha Composition
The special three image form of Masked Alpha Composition allows you use the same mask to directly merge two images together.
|
composite
" command instead of "convert
", the 'overlay' image (white parts) is given first with the 'background' image (black parts) second. In other words the first two images need to be swapped for that command.
Aligning Two Masked Images
Under Construction
On aligning two masked images... If your masks are pure boolean, you should have no problems however you apply them. However masks containing 'anti-aliased', 'gray', or 'semi-transparent' edging to make them 'smooth looking' can be serious headache if you do not handle them properly and with care. The rest of this discussion is on 'anti-aliased' masks. Anti-Aliased Masks which join together come in two styles... * Ones which fit together like jigsaw puzzle pieces OR like a shaped peg into a shaped hole (shared boundary) * Masks that are ment to overlay a solid area (layered) The latter is easy to handle and is the normal effect you get when you overlay some colored shape over a fully-opaque image. Essentially you would use 'over' composition to compose the shape. The former 'jigsaw' masks however is harder. Such masks are not meant to either overlap, or underlap each other. And yet if you try to join them using the obvious and normal 'over' composition you will end up with a semi-transparent join where 'anti-aliased edges' are merged. Example of a bad 'jigsaw mask' join (over) The correct way to join masks and shaped 'jigsaw' images is to use Plus composition to 'add' the images together, with either a black or fully-transparent background. Example of a correct 'jigsaw mask' join (plus) For another example of DIY image joining, using 'Dst-In', 'Dst-Out', and 'Plus' composition, see examples in... https://imagemagick.org/Usage/compose/#dstin I also go though this joining detail in the bug report of 3 image alpha composition Composite Mask Bug - Fixed. For more on the difference between 'over' and 'plus' see 'Blend' (plus) vs 'Dissolve' (over) Examples of correctly joining edge aligned pieces is shown in 3d Cubes - Affine and again in 3d Boxes - Perspective and in Isometric Cube using Shears https://imagemagick.org/Usage/warping/#sheared_cube The Major problems in these examples is that the individual parts were NOT generated using the same mask, but distorted to their final positions. As such they do not quite fit together properly and joined together. These examples need to be updated to use a 'Plus' composition method. To generate improved results, but even then they will still probably not be quite 'right' as the masks do not exactly 'fit' together.
Generating Correct Edge Aligned Masks
The best idea is to use the same mask (negated) for BOTH pieces, rather than attempting to draw the two masks separately. Otherwise you have the two masks overlap, OR leave a gap, exactly as you have seen. Correct methods of mask joining.. * use mask to set transparency on one piece use negated mask to set transparency of other piece 'Plus' the two pieces together. * Use mask to Add transparency to just one piece, then 'Over' compose that piece over a complete image. * use a three image masked composition see https://imagemagick.org/Usage/compose/#mask and https://imagemagick.org/Usage/masking/#masked_compose Which uses the mask to select results from two different images. Remember, 'Over' only needs the 'source' or 'overlay' image masked, the background image should not have aligned semi-transparent edges. But a 'plus' composition needs both images masked with and exact negative mask of each other align the joined edge. WARNING: Draw does NOT currently allow you to generate two shapes that will fit together properly without overlap!!!! See Draw Fill Bounds for details. I have not checked SVG to see if it has the same problem.
Special Image Masks
Write Masks - Protecting Pixels from Change
A 'write' or 'clip-mask' is a special greyscale image that is added to an existing image of the size size. It defines areas of the image which are to be classed a 'immutable' or 'not-writable' by most image processing operators. The operator "-mask
" takes an external image to be linked to the images in memory. The 'plus' form of the operator "+mask
" removes the mask from the image. For example, here I use a 'write mask' to protect the background pixels from being written to, while rotating the hues, to re-color a the foreground red rose to a blue rose.
The mask is bit rough, but it worked well. Just remember that a 'write mask' is used to specify the part to be protected or preserve. Remember, in IMv7...
The "
For more advanced example see Chroma Key Masking, which is more about generating the mask, rather that applying it as a write mask. Write or clip masks are designed to work when the pixels in an image are being directly modified. EG: negate, level, color tinting, modulate, drawing, composite, morphology, convolutions. For operators that generate NEW images (resize, distorts, extent, etc) it will fail to preserve original pixels, as the mask will not be able to correspond to the new image size. Such operations will also have the side-effect of removing or unsetting the images 'write-mask'. Here is another example...
-mask
" operator defines a 'write-protect' mask
magick -size 70x70 xc:red red_image.png magick -size 70x70 xc: -draw 'circle 35,35 30,5' write_mask.png magick red_image.png -mask write_mask.png \ -fill blue -opaque red +mask masked_color_replace.png |
This use of masking is actually exactly how Composition Masking actually works! But only for the duration of the composition operator being applied.
magick -size 70x70 xc:green green_image.png magick red_image.png green_image.png write_mask.png \ -composite masked_composite.png |
|
A three image "
-composite
" operation uses a 'write' mask
In morphology write masks are typically used to generate a Conditional or Constrained Morphology form of an operation. One such example was discussed in the IM Discussion forum, Cleaning up noise around text, to limit the effects of a dilation. NOTE: -crop should be able to preserve the image mask of individual images, by also cropping the mask and assigning to the new images. This however is currently not done.
Clip Mask and Clip Paths
The "-clip-mask
" form of this operator, is almost exactly the same as the above but only provides a boolean (all-or-nothing) style of masking. As a result you cannot achieve a 'blended' or smoothed result. For example...
|
-clip-mask
" does not produce a blended result as "-mask
" does. The only good thing about this is that it is slightly faster (though not very much). It was provided originally to allow the handling of Clip Paths in TIFF image files, and is a very old operator (IMv5). The newer "-mask
" operator should be used instead.
In IMv7, a 'write-mask' and 'clip-mask' are implemented side-by-side, even though they technically do exactly the same function. As such you could apply both masks simultaneously. However using both at the same time is not recommended, and results are not defined. Also this 'boolean mask' form has been removed from IMv7. |
Clip Paths for TIFF images
A 'clip path' is part of TIFF image file format, and defines a vector path that is used define a 'shaped area' within the TIFF image. In IM the operators "-clip
" and "-clip-path
", reads this 'clip-path' and converts it into a Clip mask (above). As such it defines a 'write mask' that will protect the shape from modification. A clip-path stored in the TIFF image is defined as a SVG Path Drawing, which you can extract from a TIFF image file format using...
magick identify -format '%[8BIM:1999,2998:#1]' image_clip.tiff |
The biggest problem people often have is making everything that is not clipped transparent. Which requires you to write the areas the mask write protects! This is one solution, which converts the whole image to transparency, then turn on the 'clip path' then make the now writable parts opaque (visible) again.
magick input.tiff -alpha transparent -clip -alpha opaque -strip out.tiff |
+clip
" operator also turns off and remove the clip mask (just as "+clip_mask
" does). However no file format saves the current clip mask with the image for any image file format. (At least in IMv7)Read Masks - Ignore Pixel Input
It is important to note that a write mask will limit what pixels will be written to an image. It does not however limit what pixels are being 'read' as part of the operation being performed, to create the new pixel data being written. This basically means that if you use a 'area effect' or 'neighbourhood' type of operator, such as Blurs, Morphology, or Convolution, then 'writable pixels' close to the edge may include color values from the masked, or un-writable area. For instance, here we write protect the foreground rose, before blurring the image. That is, we want to only blur the background part of the image, rather heavily in this case.
magick rose: -mask rose_fg_mask.png \ -blur 0x8 +mask rose_bg_blur_fail.png |
The result is using a Write-Protect Mask and is not what was wanted.
Read Mask Solution for IMv7
In IMv7 the only way to make a pixel color pixel unreadable, is to make the pixel transparent. Transparent pixels have no color by definition, and as such the 'hidden color' is not part of the calculations made by the blur operation. This gives us a 'cheat'. Make foreground pixels transparent, apply the blur (or other) operation, and turn off transparency (it isn't really wanted in this case). Then we can restore the foreground part of the image. If that sounds complex, it is. Here are the steps involved, while showing intermediate images to try and make the technique clear...
magick rose: rose_bg_mask.png -alpha off \ -compose CopyOpacity -composite +compose rose_bg_only.png magick rose_bg_only.png -channel RGBA -blur 0x8 rose_bg_blurred.png magick rose_bg_blurred.png -alpha off rose_bg_blur_opaque.png magick rose_bg_blur_opaque.png \ rose: rose_fg_mask.png -composite rose_bg_blur_good.png |
Write |
Read |
Masking Method Differences |
Regions and Region Sub-Images
Regions are another way of limiting the effects of operations to a smaller area of an image. For example, here I color tint the whole rectangular region red...
|
|
Before IM v6.6.9-5 transparency preservation was broken, and the results of transparency in a region was always "see-thru to original". As such the result of the above would not include any transparent pixels, even though the image allowed the use of transparency. |
Warping a Local Region
As a 'image region' actually extracts a 'small sub-image' of the original for processing, you can make use of the special 'localised' Circular Distortions to warp small regions of the original image. For example, here we have a line of stripes.
magick -size 600x70 xc:darkred \ -fill white -draw 'roundrectangle 5,5 595,65 5,5' \ -fill black -draw 'rectangle 5,25 595,31' \ -fill red -draw 'rectangle 5,39 595,45' \ lines.gif |
magick lines.gif \ -region 90x70+10+0 -swirl 400 \ -region 90x70+100+0 -swirl 400 \ -region 90x70+190+0 -swirl -400 \ -region 120x70+280+0 -implode 1.5 \ -region 100x70+380+0 -implode -7 \ -region 101x70+480+0 -wave 10x50 -crop 0x70+0+10\! \ +region lines_regions.gif |
-implode
" and "-swirl
", fit into the use of regions very well, as they have the property that the outside edge of the distorted image matches up to the rest of the image outside the defined region. That is, they are actually designed to perform 'localized image warping'. Note that when I used the Wave Distortion, I had to crop the size of the resulting 'wave' image so that it would again fit into the original area from which it was extracted. Remember Regions only work when used with Simple Image Processing Operators. Any other operator including another "-region
" operator will cancel the region processing, before that operation is applied.
How Regions Work, and its Problems
In reality the way regions work is...- Extract from the image a smaller image according to the "
-region
" operator, using a simple crop with the region argument. - Apply any Simple Image Processing Operators, that follow, to the smaller image.
- When a non-Simple Image Operator is seen, OR another "
-region
" operator is found, OR the region is turned off using "+region
", then the extracted region is overlaid on the original image at its extracted location.
... -region WxH+X+Y ...simple-operators... +region ... |
... \( +clone -crop WxH+X+Y ...simple-operators... \ \) -geometry +X+Y -composite ... |
... \( -clone 0--1 -crop WxH+X+Y ...simple-operators... \ null: +insert \) -geometry +X+Y -layer composite ... |
|
|
Region Images which are enlarged or shrunk, may not 'fit' back into the original. For example, here I resize (and color) the region image so it becomes smaller...
|
|
Like "mogrify " you cannot merge multiple sub-images as that requires the use of a non-simple image operation. However you can use "-draw " as an alternative composition method. See Alpha Composition in Mogrify for an example. |
At the time of writing, the 'region image' still contains the Crop Virtual Canvas Offset from its extraction from the original image. This may, or may not be regarded as a bug, depending on if you find this information useful or not. The offset is currently not used when a region image is restored. If the offset is not wanted (as it interferes with an operator such as Distort), follow the " -region " option by a "+repage " operator to remove the offset, from the region images. Its removal or modification will not effect its restoration back onto the original image. |
Background Removal
One of the most common problems in image processing is mask generating from a existing fully-opaque image. Such images are commonly downloaded from the World Wide Web, or generated by programs, or in image formats that don't provide any form of transparency. It may also be that you have a photo of some object, and want to remove the background. Remember photos do not have any understanding of transparency, so you need to remove the unwanted parts yourself. Unfortunately there is no general solution to this problem, especially when you also want to retain any semi-transparent edging to the image. Consequentially their are hundreds of ways and variations on doing this task, all dependant on the exact situation. Closely related to image masking is transparency adjustments to match a background that an image is going to be overlaid on. This is talked about in detail as part of saving to the GIF Image File Format which only allows Boolean transparency.Masking Simple Backgrounds (floodfill)
When an images background is a simple single solid color, you can often generate simple masks (and background removal) by just doing Replacing Colors in Images. For example, here is a direct floodfill masking of an image with a solid color background.
|
|
|
|
Cutting Out Bordered Objects
Images with an existing single color border has a distinct advantage for these methods of background removal, as the border provides a definite boundary between what is 'inside' and what is 'outside' the image, and that in turn allows use a better method of specifying the boundary of the background image. That is, rather than specifying what colors should be regarded as background, we can instead specify what colors mark the border of the object being masked. Further more, as the border color is known, only two specific colors will have been mixed together around the edges of the image. That is, both colors are known, and so exactly how transparent the edges should be is also very well known.
Under Construction
Removing a Known Background
While removal of a simple background to a 'Boolean' mask, is relatively straight forward, things get more complicated when the background is not so simple. However if the background itself is known. you can use that to help in its removal from other images. As of IM v6.3.4 a special Alpha Composition method was added called 'ChangeMask
' which allows for the direct removal of a known background from an image. For example, here we have an unaltered background image, and one that has been overlaid by a GIF image with a simple Boolean (straight on/off) transparency. By using 'ChangeMask
' we can recover that original overlaid image (if it is very different to the background).
Basically what this does is determine how 'different' the pixels are from one image to the other, and is the difference is less than the current Fuzz Factor, then make that pixel transparent. Only fully transparent pixels are added to the image, otherwise the original image is left as is, transparency and all. We can simulate the operator by using the older 'Difference
' composition method to generate a Comparison Difference Image...
As you can see the difference image is black for all the unchanged parts and a mix of colors for the parts which has changed.
Using this mask we can set anything that has not changed to transparency.
|
ChangeMask
' composition method makes this process a lot easier. However this only presents a 'on/off' style of background masking. It does not allow for fuzzy or anti-aliased edges, or transparent feathering of the result.
Difference Image Masking and Feathering
The above can be taken further to images that have aliased edges. as well as non-simple backgrounds. For example, Here we have a 'Cyclops' on a white background, which we want to extract. We then generate gray-scale image of the differences between this image and the background color (as defined by top-left most pixel).
|
|
-threshold
" you can add a 'fuzz factor' to the boolean (on/off only) mask, so as to get the mask closer to the image proper.
|
|
-blur 0x0.707
" or square rot of 2) is actually recommended when generating threshold masking, just to smooth out the edging of the mask. Of course the result will not be boolean, so don't try to save it to a GIF format image file. This is also an example of Blur Feathering. But be warned that it is not quite the same as true Feathering Shapes using Distance. However when dealing with 'bitmap' or 'threshold masks' such as we created above, a small amount of blur feathering, followed by a larger amount of distance feathering, will probably result in the best overall result.
Recovering Semi-Transparent Edges
The Difference Masking technique that we used above can be used with the previous FloodFill Masking technique to solve most of the problems we have seen with simpler masking techniques. Here we look at a multi-layered masking technique, but one that should produce near ideal removal of the images background, while preserving the anti-alias shading pixels along the edge. However this is limited to images on a known background, and having a good contrasting 'edge' to the foreground pixels. For this example I decided to use something that was very hard to separate, but which showed a LOT more shaded pixels around the edges than you would typically have for anti-aliasing purposes. A shape with a shadow effect.
|
|
|
-fuzz 1% |
-fuzz 3% |
-fuzz 6% |
-fuzz 28% |
-fuzz 32% |
-fuzz 34% |
1%
' which still contained a large area surrounding the image. In a more typical non-shadowed case this area can be even smaller, down to a non-percentile value such as 5 or 10. The second mask should have large enough 'fuzz' so as to eat up all the semi-transparent pixels that is present. That is, right up to and preferably actually into border of the image without completely removing the border, or 'leaking' into the image proper (see last image above). The negative of this mask will actually represent all the pixels that will be fully-opaque (and thus represent the inside) in the final image. This selection can be difficult and may require a lot of trial and error to figure out the best value to use. For this image a very high fuzz '32%
' was able to be chosen without any major problems. Basically you want to try and get it high enough that the final image will not contain any of the original 'background' pixels in it, but without the mask eating away (or leaking into) the inside the image. It may even require a little hand editing to get the mask just right when you have a gap in the surrounding 'edge' color. We can now use this mask to extract the 'core' or inside of our image. That is the parts we are sure does not contain any semi-transparency through to the background pattern we are removing.
|
|
|
You will need to somehow figure out what color the semi-transparent pixels should be, so you can set the correct color for the anti-aliasing pixels. This could be
|
|
|
|
bg_removal
", which uses a single command, no temporary files, and has a number of extra options on the methods by which the masking is performed.
Background Removal using Two Backgrounds
The major problem with the previous techniques is that you really do not have enough information to completely recover all the information about the foreground object. You really need to recover two pieces of information, how transparent each pixel in the foreground object is, and what is its original color. And you can not perfectly recover both pieces of information from just one image. Even when you know exactly what the background image looks like, you cannot just subtract it from the foreground object, unless the two are very different and known colors. The problem is you simply cannot be sure if the color that is visible is really the color given (opaque), or it is some blending of some other color and the background (semi-transparent). You cannot separate the original color from the alpha value needed, unless you have a source of some extra information. The one situation in which you can completely recover all the details of a foreground object, is when you have two images containing two very different but completely known background colors. In that situation you do have enough information to recover both the color and its transparency of the foreground object, for a perfect background removal. The important factor in selecting two images, is that the background colors are as different as possible over the whole image. That is, the colors are not only color complementary, but negative in intensity in all channels. For example... While a different background color is used, both images contain exactly the same object. The object shown is not simple, but contains lots of semi-transparent colors. You can see this in the way the dark blue background is visible in the flames of the image, though this transparency is all but invisible in the lighter yellow background. By using two colors, the semi-transparent pixels of the overlaid object will become mixed with two very different colors, as a result be slightly different colors in the two images. By measuring how different each pixel is, you can determine exactly what pixels are semi-transparent, and by how much. Essentially there is enough information to allow you to perfectly recover the transparency of the overlaid object. Recovery the transparency or 'mask' is of course the first step, and is actually a very straight forward step. Generate a difference image, then merge and maximize the differences found in each channel.
|
-evaluate-sequence max -auto-level
") in the above , you will need to divide each channel by the difference of the two background colors. That is, divide by a value between 0.0 and 1.0, the larger the difference the better. If the two background colors is pure black and pure white, then no normalization, is needed, just the difference of the two images. The difference is then Negated so tha a maximum difference produces zero alpha or full-transparency, and no difference produces maximum alpha or full opacity. The next task is harder, as the colors of each semi-transparent pixel is modified by the background, you cannot just use the alpha mask to extract the object from one of the source images. For example...
|
|
u.p{0,0}
') in the source image as the background color to remove from semi-transparent pixels. Adjust this or directly substitute the color to remove if needed. The key to the color restoration, is the complex FX blended subtraction operation in the above. This will enhance the original color ('u
') of the source image according to the alpha mask ('v
'), then subtract the background color (u.p{0,0}
or the top-left corner pixel) from the final result. The formula is not straight forward, and major thanks does to HugoRune, in the IM Forum discussion Undo a Composite -dissolve for determining the mathematics needed. The discussion also goes on to exactly how all the steps work, were derived, and even how you can also extract the overlay from any two known but different background patterns. Here is the whole sequence all in one command.
|
In IM v6.6.8-3, if FX references a transparent pixel using 'p{}' it gets zero values rather than the actual fully-transparent color values! This is a bug and was reported and fixed in IM v6.6.8-5. It is not known when the bug was introduced. This was only a problem, if you decide to merge the alpha image into the source image first, then try to fix semi-transparent or 'spill' colors, using the known background color. |
0
' in the second "-clone
" operation, but either source image could have been used. Just one warning. The above assumes the top-left pixel is the unadulterated background color. If it isn't you may have to modify the command to specify a specific pixel color, or use a third image that does contain the correct background color information. The latter method is vital if the background color is not constant across the image, though even that complication can be fixed. Here is the simpler sequence for images overlaid on a pure black and a pure white background color. In this case the colors are always recovered from the black background image, as it is just a simple division, and thus a faster Divide Composition can be used instead of the painfully slow FX DIY Operator.
magick match_black.gif match_white.gif -alpha off \ \( -clone 0,1 -compose difference -composite -negate \) \ \( -clone 0,2 +swap -compose divide -composite \) \ -delete 0,1 +swap -compose Copy_Opacity -composite \ match_recovered_3.png |
Studio photos for background recoveryThe ideal backgrounds are a matte (non-reflective) black, and simple pure (non-reflective) white. The background should also be as smooth and unvarying shade as possible. For making photos specifically for background removal, using two complementary colors may work better. Say taking photos with a green and magenta backdrop. Basically you will need to some how replace the background color screen before taking the second photo. Note that order of the two photos does not matter in the background removal, but they should be a clean and uniform as possible, and the primary object and camera must remain perfectly steady and fixed. A better method may be to simply place a white screen well distanced behind the object and uniformly light that screen using two different colored lamps so as not produce any shadows from the object. With this technique you can switch to the other background color, without needing any physical change in the studio to take two photos with two different backgrounds. These two color background techniques should work well for transparent objects, but reflections, and or background warping or 'lens' effects by the object being photographed will not be recorded by the technique, only its transparency. On the other hand reflections of a constant light source on the object will be preserved! If you try this please let us know, and give an example of your source photos and results for inclusion here. You will be named with a pointer to your site for people to look at.
Video background recoveryIf you have a large enough series of images with many different but complex backgrounds (such as a video), you can try taking a minimum and maximum value of all the images to generate a near pure black and white background image for use. The more images, the better this works. With those two images any constant logo and its semi-transparency can be extracted, and the same technique can then be used to remove it from all the frames. However it will only work for a constant semi-transparent overlay, and may not work for logos that use color or hue distortions, or even a solid color logo. But it will will at least let you determine the exact logo shape. For logos that are fully opaque or more difficult, hole filling (see next) can then be used to fill in missing detail from surrounding colors. See the IM Forum Discussion for more details.
Hole Filling
While masking, adding transparency, and removing background provide one way of dealing with unwanted elements, often a 'hole' is not what you actually want as a result. Sure you can just overlay images with holes over other images to fill them, but that may not provide a seamless result. To erase elements from an image, you don't just want to cut them out, but replace them with colors, shades, and textures of parts that surround the hole. The following are various techniques for determining what to use to fill-in that hole.Creating a hole to fill
Suppose we have an image with some ugly text...
|
For this case however I'll create a mask using a drawn line that covers the 'ugly text', as if an user had quickly used an image editor.
|
Now lets use the mask to cut a hole out of the image, which will also check that it covers all the unwanted parts.
|
Blurred Fill
So we have a hole, that needs to be fill with some color. Something that will not look like we have actually removed somethign from the image.
One of the simplest methods is to simply blur the image, allowing the colors around the hole to 'spread' into the hole, and then remove the transparency.
|
magick zelda_text_hole.png zelda_text_fill.png \ -compose Dst_Over -composite zelda_text_removed.png |
Under Construction
Links to other methods. Resize blurring hole filling method... Sparse Color, Shepard's Method (fast). See also snibgo, Filling holes Blurring Edge Pixels only...Sparse Color as a Fill Operator. See also snibgo, Filling holes in priority order I would like to use a morphology operator that sets color in the color channels while working out distance in a hidden background channel. This should generate a very fast no-leak Shepard's like fill known as, 'Color Diffusion'. See the paper Diffusion Curves which makes heavy use of this technique. A large and old discussion on hole filling (text removal) is on the IM Users Forums Text Removal Discussion. A newer discussion is Fill area with nearest colour from boundary, which is more about filling without blurring. Some other non-IM methods of 'hole filling' to erase parts of images is shown on Stack Overflow, Remove text from jpeg. For example Using Python Skimage. Or using Python OpenCV inpainting