Pulling objects out of an image

The term cutting has various meanings in the field sof photography and image editing. Photographers understand that it can mean images that have been taken with an open aperture so that they have less depth. This causes the theme of the image to appear sharp while objects in the foreground and background are blurred. They are pushed back and become part of the background.

In image editing circles, the term cutting frequently refers to cropping the desired object down to size. This procedure is easily handled by an image manipulation program. Difficulties do arise however when the term is used to refer to the extraction of an object from a current background so that it stands against a transparent background. This extraction function is important if you want to work on any type of collage. Therefore, if you want to extract an object from one image and place it in another, you will need to be able to use this function.

If you have ever tried to extract an object from an image, you probably know the problems that can occur during the procedure. Images taken with a digital camera almost always have areas in which the object and background seem to melt into one another (Figure 1). Closer observation reveals that there is no sharp delineation between the object and the background. Many of the details flow into one another. Automatic processes like color choice and magnetic scissors do not work reliably here. Another related problem is that the edges of the object that has been extracted need to be edited so that when added to a new image the object has edges that melt seamlessly into the new background.

Figure 1: Although the image was developed from a RAW negative, the foreground and background are optically difficult to separate. In JPEG images, artifacts worsen the result even more.

Standard Methods

The open source image manipulation program GIMP offers five standard methods for clean removal of objects. A further possibility is found in the GMIC [1] extension collection in the form of Extract Foreground [2] . Table 1 summarizes all of the options and states the advantages and disadvantages of using each. In addition, there is also the specialty Seamless Cloning tool (see the "Seamless Cloning" box).

Table 1

Standard Methods

Methods Advantage Disadvantage
Integrated into Gimp
Selection Especially simple and quick Margins need to be reworked
Quick masks Permits precision editing Lots of effort
Erase Quick, relatively precise, margins removed well Limited application
Masks Relatively precise, extensive possibilities Lots of effort
Channels Very good results The most complicated method
Using GMIC as an Extra Tool
Extracting Quick, often sufficiently precise Margins need to be reworked

The selections and the quick masks generate a precise selection of the object that is to be extracted and then transferred via copying and inserting into a new layer. There are basically two possibilities for working with selections. You should try to either create a selection with maximum precision, or use a rough selection. The first option corresponds to the selection method. The second represents the basis of a different procedure.

Generally, copying into a separate layer is available in all methods. If the object that is to be extracted has a shape that is not suitable for one of the geometric selections, then you should opt for the lasso, and thus freehand selection. A copy of the selected area is created with the usual key combinations Ctrl+C and Ctrl+V. The "floating" selection is then transferred via layer | to the new layer or via the first button in the layer dock to a new transparent layer. Alternatively, you can directly add the clipboard as a new layer with edit | insert as | new layer .

Seamless Cloning

With its Seamless Cloning tool, Gimp [5] provides you with a feature that is eminently practical. In order to use this tool, you select an object freehand and copy it into the clipboard. Then the tool should be activated in the target image so that the object can be inserted into the current image. The tool lets you move the object around on the background layer. Pressing the enter key anchors the object in place. At this point, Gimp uses the area surrounding the object to harmonize the brightness of the background layer with the contents of the clipboard so that a seamless match ensues.

However, the implementation of seamless cloning contained in the current Git Master does not yet work satisfactorily. Presently, areas that you want to paste in cannot have any holes and must consist entirely of a contiguous whole. Inserting long, thin areas does not work either. In addition, the paste in action has to be performed in such a way that all of the area appearing in the clipboard also appears in the image.

However, the biggest hurdle with respect to the practical use of this function is how slow it is. As soon as you transfer larger pieces, the insertion, or pasting, can only be performed with lots of computing time. If the relevant algorithm ever takes off and the developers are able to solve the issue, then Seamless Cloning would probably be a viable alternative to the methods we describe elsewhere in this article.

The Selection Methods

As a rule, you should rely on the lasso tool or free selection when using the selection method. One advantage to the lasso tool is that it allows you to connect larger areas with straight lines. Another advantage is that you can create a soft outline of the shape by pressing the left mouse button intermittently (Figure 2). When you finish drawing with the tool, the lines come together in a rough grid once the Control key is pressed simultaneously.

Figure 2: With the free selection tool, you can to use the click and free selection modes together with one another. Precision is not required. For detailed work, you can later apply the eraser.

If instead of the lasso, you choose the color selection tool, then the correct choice of parameters becomes very important. Figure 3 shows the numerous settings available in Gimp 2.9.3. It is easy to overlook the especially important selection after . This determines how Gimp should interpret the colors that have been used for the selection. The default setting Composite adds the RGB values even though Hue , Saturation , and Value are often the better criteria to use.

Figure 3: The program is designed to provide an especially large variety of selections. Optimized settings lead to very good results.

The "intelligent scissors" tool is an alternative to freehand selection. This tool has a useful extension, the Interactive boundary , in the previous version of Gimp 2.9. This extension makes sure that from the beginning, Gimp shows how moving a control point influences the line (Figure 4). It is often convenient to have this information since the intelligence for the scissors has not been around very long. It frequently results in errors, even when the borders are clearly visible.

Figure 4: The magnetic and intelligent scissors make it possible to manually adjust selection lines that have been created automatically. However, the tool does not work very reliably.

Gimp knows lots of ways to correct the first, rough selection. You can supplement or reduce the selection by one selection by applying the customary techniques of holding Shift for adding and Ctrl for subtracting. Alternatively, the option Feather lets you soften margins. In case a selection has already been made, edges can be softened with Select | Feather … . You should remember in the process to define the double Radius for Gimp to use.

You correct the current selection of pixel values with Grow and Shrink . These options are also found in the Select menu. Typically you only need specify a few pixels that Gimp then uses for the entire selection. The script Remove Holes [3], which is already built into Gimp 2.9.3, also automatically closes almost all of the small holes in the selection.

Quick Masks

Quick masks provide a means by which you can easily correct existing selections. You activate this mode by clicking with the mouse in the lower left corner of the work area (Figure 5). Alternatively, you can select the mode from the Selection | Toggle quick masks menu, or by pressing the Shift+Q keys.

Figure 5: The quick masks mode lets you edit selections with drawing tools. The button takes the user to a pop-up menu that contains various options.

Afterwards, GIMP overlays a red layer on the image for which no selection has yet been made. You work on this layer with a drawing tool such as a pencil or paintbrush . Gimp will then create a selection in areas that are painted with a white color. When you use black, this removes the existing selection. Gray tones are used to create a detailed gradient.

If the color of the virtual film clashes with the image that is being edited, it can be adjusted via the pop-up menu found under the quick mask button. As soon as this "tracing paper" matches what you need to select, you click on the button with the mouse, and Gimp will once again display a normal view. Since the editing program when operating in this mode only displays the 50-percent line of the selection, the detailed scaling does not appear any more on the screen. It does persist, however, as becomes apparent with subsequently copying.

After completing work on the selection, you should not forget to transfer the relevant, or selected, parts of the layer to a new layer. This has the advantage that you create a transparent layer. One of the special characteristics of these types of layers is that they contain all of the information that was found in the original layer. Therefore the layer that has been copied partly consists of pixels positioned on spots where the layer appears to be transparent. At this point, the erase method becomes handy. This method will be discussed later in the article.

One version of this selection method consists of applying paths. In Gimp, the procedures for converting paths into selections and selections into paths along the 50-percent line are very similar to one another. You first create the paths along the lines intended as selections. Then, you take the path tool and create a closed linear shape in Design mode.

The advantage of this tool is that the control points can be moved at any time. Likewise, it is possible to adjust the segments, which are the lines running between the control points, in terms of position and shape. However you pay a relatively high price for these freedoms. Creating a selection in this way takes a lot of time. Even so, this tool offers elegant solutions for many selection problems, especially when an asymmetrical shape is involved. Figure 4 shows good examples with the sail and the hull.

The Eraser

It is much easier to perform an extraction with the eraser. This tool offers suitable erasers for removing all of the undesired parts of an object from the current layer. In order for this to work at least partially well, you first have to prepare the tool in accordance with the task at hand. You should select a suitable softness, specify a proper brush size, and then check to see whether the other parameters are correct (Figure 6).

Figure 6: Normally, erasing works by first moving from the outside to the inside. If the user removes too much from the image, the action can be undone with Ctrl+Z.

Now you work from the outside moving to the inside. Sometimes it happens that too much material is removed. At this point, the transparency property becomes useful. The anti-erase mode can be activated via holding down the Alt key. In this mode, Gimp restores the pixels that were previously present in the area. Unerasing is typically performed by moving from inside the selected object to an outside border that is no longer needed.

With the soft brush points (Hardness … ), the brush size also influences the soft area. It makes sense therefore to define a key combination that can be used to quickly change the size of the brush. The coverage achieved by the erasing tool also influences the strength of the effect. It can be helpful to work with less covering power. However, there are only a few cases in which this technique is actually advantageous. You can usually achieve a faster result with layer masks.

In order to make the work somewhat easier in this mode, you should lay a monochromatic layer under the work layer that was previously copied. This is done in order to create a better contrast and increase visibility of the object. Then you can immediately figure out where too much or too little has been erased. If necessary, each layer can be converted into a transparent layer via the Layer | Transparency | Add alpha channel menu or via the pop-up menu for the relevant layer.

Layer Masks

Many image editors think of layer masks as the silver bullet of extraction. The basic idea behind these masks is simple. The user defines an additional channel for each layer. This channel then controls the visibility of the pixels in the layer. In theory, this corresponds to the procedure with the alpha channel. However, Gimp gives greater emphasis to the support for the layer masks since they offer other possibilities as well.

Gimp lets you add one layer mask to each layer whether it is transparent or not. This layer mask addition works via the menu under Layer | Mask | Add layer masks … (Figure 7). Gimp creates a narrow border for both the layer and its mask in the layer dock. The active part is highlighted in white. Clicking on the corresponding preview image toggles the action. As a result, you can look either at the layer itself and how it is influenced by the mask, or the layer mask.

Figure 7: You can recognize layer masks by the double symbol in the layer dock. The active part is highlighted in white.

In the dialog for setting up the mask, you specify how it should look. The standard Gimp mask is pure white. This corresponds to full covering power. You will not see any difference with the layer applied from the original state. With a pure black layer mask on the other hand, you will see nothing at first. By adding white color to the layer mask, you then start to see the objects. The airbrush tool is especially well suited to the task in this mode.

The other possibilities are better suited to special applications. Layer's alpha channel uses the contents of an alpha channel to create the layer masks. Transfer layer's alpha channel works in a similar fashion but reverses the effect. Only the layer masks control the result. If a selection already exists in the image this can be accepted via Selection as a layer mask. Gimp translates the brightness of the image into the covering power of the layer with Grayscale copy of layer . Dark areas disappear; light areas remain visible. The channel option is more interesting. It detects the layer masks from the brightness similarly to the grayscale copy. However it uses the color of the selected channel. This option does not work with all Gimp versions.

Many times, you will want to create a layer mask by itself in order to have complete control over the results. There are many ways of doing this. Often, the current layer can serve as the basis for the mask. You can create a layer copy without a mask and edit it in such a way that, ideally, the motif becomes either completely white or completely black while the environment has the exact opposite color tone. Afterwards, you copy the layer that has been prepared for the mask to the clipboard and activate the mask from the work area.

However, Gimp displays the result in the layer dock in a somewhat confusing way. The floating selection appears as usual above the layer that has a mask. Gimp is able to apply the layer as a layer mask according to your wishes only when it is anchored with the anchor button underneath the layer dock for this layer. Once this is accomplished, there are additional possibilities in the context menu of the layer dock for interacting with the layer mask. For example, you can cause Gimp to display the mask instead of the layer content. You can also temporarily deactivate the mask.

There are many factors that go into the way in which a layer mask is created. Instead of manually converting a layer copy into black and white, you can go to the Desaturate function found in the Color menu. The developer version Gimp 2.9.3 expands the capabilities of this function. Gimp now collects options from earlier versions under Desaturate… in the Desaturate sub menu. Additionally, the user will still find Color gray… , Mono Mixer… (the monochromatic channel mixer) and also a tool labeled Sepia… .

The Levels tool (Figure 8) is a likewise practical method. With it, you can control which parts of an image appear white and which appear black by means of two slider bars. These extreme contrasts in color often help you find edges for the borders of an image. In ideal situations, the Levels tool suffices for tasks like masking an almost white sky. In real life, ideal situations do not occur very often.

Figure 8: The Levels tool creates very hard transitions for masks.

There are other methods available. For example, before resorting to the Levels tool, you can bend the color of an image by using scale curves, found under Colors | Curves … in order to create the desired effect (Figure 9). Alternatively, you can go to the channel mixer under Colors | Components | Channel mixer… , or you can use the representation created using the Levels tool as a basis for manual editing.

Figure 9: Sometimes bold curves suffice to create the essence of the mask.

You can see from Figure 8 that the contours of the sailor stand out sharply from the colors of the environment. This is an optimal situation for removing the head from the surrounding water. However the method is not suitable for all areas of the test image. For example, the sky and the sail are no longer easily separable. Here it is necessary to mask the objects with paths.

Professional image editors on the other hand often use manual tools. There are many of these available and they offer many advantages. You only use black and white for colors. The pay off here is that you can apply processes and smear the colors, perform pencil in and whatever else might be needed. If you are experienced using of brush points, you can use pencil and brush for precision extraction of objects.

Gimp 2.9 has a brush named Angular box 2 [4] that adds a special capability. This is an animated brush point, and it can be recognized by the file extension .gih . The special feature of this brush is that it turns itself following the direction you take when drawing.

Extracting Hairs

Extracting extremely tiny details like individual hairs is an art [6]. The methods described here let you perform this task, albeit with considerable effort. As a rule it is necessary for you to create and adapt the masks for situations involving manual work.

Sometimes though, fine grained extractions are not necessary. There is an online Photoshop tutorial that shows a method that achieves a similar result with less work. With the exception of the key combinations, the steps described in the tutorial also apply to Gimp.

One outstanding issue related to the removal of (half) transparent structures is that the background disappears since light shines through the structures. You have the choice of either incorporating the desired transparent object in front of a completely monochromatic background or manually reworking the spots.

The Channel Version

It may be difficult to overlook the Gimp channels, but it is usually only the experienced user who employs the possibilities they offer.

However, channels offer great capabilities. They underlie the RGB(A) structure of the image by making each individual component available. Since components of an image are involved, the channels have only one dimension consisting of component brightness that exists in addition to the X and Y values. In this way, they are equivalent to very special grayscale images of the original image.

When a situation involves images where the objects and background in one of the channels are easily distinguishable, you should not shy away from using these as a selection (Figure 10). You simply place the desired channel in the layer dock. It will appear there as a new layer that can now be manipulated. As described above, a mask is used for the manipulations.

Figure 10: Individual color channels can also be used as masks.

Fine Tuning

Due to edges that can appear on objects, most experienced observers recognize the presence of inserted material when looking at images created with extraction methods.

Often times the edges look too harsh. This leads to a clear delineation between the object and the background, thus making the image look unrealistic. Frequently it suffices to soften the image on a separate layer. This is done by first creating a selection of the edges of the object via Selection from alpha channel . Then you trace these edges via Trace selection (Select | selection editor ), in addition to an eraser that has been suitably prepared. The best way to proceed is to use a soft brush point that is the size of a small area of pixels.

The flip side of this problem is that edges can sometimes appear too soft. This lets underlying material shine through. If this occurs, you should edit the edges with the eraser in erase mode. This is done by holding down the Alt key. This also activates corresponding options in the eraser tool settings. For other situations involving soft edges, you can achieve better results by editing the problematic spots with different tools, such as clones.

Conclusion

Gimp offers multiple functions and tools for extracting objects from images. As is often the case, the devil is in the details. By using the methods presented here, you can achieve fairly good results provided you are willing to make the effort. Those users convinced that extraction is easier with Photoshop will find any number of relevant Gimp tutorials online that show otherwise. These two image manipulation programs require similar amounts of time and effort to extract objects from an image.