Background Masking Filter

Calculates a mask based on specific criteria as a means of determining which pixels belong to the background and which do not.

Category

Mask and Selection

Node

Parameters

Mode: how to calculate the value that is compared against the specified threshold:

Average: all channels of the input are averaged and that value is compared against the specified threshold

Distance: a distance value is calculated against one or more references (see below), and that distance is compared against the specified threshold

Comparison: what comparison operation to use when comparing against the threshold. If the comparison is true for a given pixel, that pixel is considered to be a foreground pixel. If the comparison is false, it is considered to be a background pixel. The possible comparisons are Lesser, Greater, Lesser or equal and Greater or equal.

Distance Method: if Mode is Distance this specifies the method by which the distance is calculated. The following methods are available (they are a subset of the possible distance calculation methods provided by the Distance classifier filter):

Cosine: a scalar product is computed between the input data interpreted as an abstract vector (using the channels as the index) and the corresponding reference data. (See below.)

Euclidean: the Euclidean distance is calculated between the input data and the reference data.

Reference: how to obtain the reference to compare against in Distance mode. The following options exist:

Scene: the first N lines of the data that is being processed are averaged to obtain the reference

Groups: any group that is selected as applicable to this filter will be averaged (individually), and the distance will be calculated for each individual group. As long as one of the distances meets the threshold, the pixel is considered to be a foreground pixel

Applicable (group-specific parameter): if Mode is set to Distance and Reference is set to Groups, this parameter indicates which groups should be considered background references.

Parameters (Continued)

Column Mode: how columns (x direction of the image) should be treated by this filter:

Global: no distinctions are made between columns. If Reference is Scene the reference will be the same for all columns and will consist of the average of all pixels in the initial N lines.

Exclusion Zones: the manner in which the reference is obtained (if Reference is Scene) is identical to Global, but the way the threshold is applied is different. In the first N lines of the scene the filter will detect columns that will never match the background condition and exclude them from further comparisons. (See below for a detailed explanation.)

Individual: columns are treated individually from each other. This is only available if Reference is Scene, in which case a background reference is extracted for each column individually. In that case it is recommended to use more lines for the initial refernce height, since in the other modes the background reference will already be averaged over the various columns to reduce the effects of noise.

Threshold: the actual threshold value to compare the average and/or distance against.

Scene Reference Height: only available if either Reference is set to Scene or Column Mode is set to Exclusion Zones (or both). Specifies the number of rows N that should be taken into account when either obtaining the background reference and/or when determining which columns to exclude.

Extension Size: by how many pixels/columns to extend the exclusion zones in each direction after they’ve been determined.

Inputs

Input: the input to use to determine the mask

Outputs

Mask: the resulting mask

Effect of the Filter

This filter is most useful for separating background from foreground pixels. The filter genrally operators separately on each pixel, with some exceptions for the most advanced modes.

The filter always outputs a mask that may be used in conjunction with a Mask-based selection filter to mask out the background pixels from the input. A mask may also be used as the input to a Mask-based object detector filter.

Average Mode

If the Mode parameter is set to Average then all channels of an input pixel are averaged, and the resulting average is then compared to the provided Threshold value via the operation selected in the Comparison parameter. If the comparison is true, then the pixel will be considered a foreground pixel, if it is false, a background pixel.

For exampel, setting Threshold to 0.5 and Comparison to Greater will lead to all pixels whose average is greater than 0.5 to be classified as foreground, and all other pixels to be classified as background.

If Column Mode is Global then the effect of the filter is identical to the effect of an Instrument function filter set to average all channels, followed by a Threshold filter to actually perform the thresholding.

Distance Mode

If the Mode parameter is set to Distance the filter requires a background reference it may use to compare the input data to. A distance is then calculated between the input and the reference, and that distance value is then compared against the Threshold value provided by the user.

The Distance Method specifies how the distance is to be calculated. For the purposes of the following formulas, the input channels of each pixel will be interpreted as an abstract vector $\mathbf{y}$ , and the reference as an abstract vector $\mathbf{y}_{\mathrm{ref}}$ .

In Cosine mode the distance will be given by the following formula:

$d = 1 - \left| \frac{\mathbf{y} \cdot \mathbf{y}_{\mathrm{ref}}}{\left\Vert\mathbf{y}\right\Vert \left\Vert\mathbf{y}_{\mathrm{ref}}\right\Vert} \right|$

For input data with three channels, this will correspond to the formula:

$d = 1 - \left| \frac{y_1 y_{\mathrm{ref},1} + y_2 y_{\mathrm{ref},2} + y_3 y_{\mathrm{ref},3}}{\sqrt{y_1^2 + y_2^2 + y_3^2} \sqrt{y_{\mathrm{ref},1}^2 + y_{\mathrm{ref},2}^2 + y_{\mathrm{ref},3}^2}} \right|$

This is often also called Spectral Angle Mapper (or SAM) in the literature.

In Euclidean mode the distance will be given by the Euclidean distance between the input and the reference:

$d = \left| \mathbf{y} - \mathbf{y}_{\mathrm{ref}} \right|_2$

For input data with three channels, this will correspond to the formula:

$d = \sqrt{\left( y_1 - y_{\mathrm{ref},1} \right)^2 + \left( y_2 - y_{\mathrm{ref},2} \right)^2 + \left( y_3 - y_{\mathrm{ref},3} \right)^2}$

Background References

When Mode is set to Distance a background reference is needed. There are two options for this mode:

Groups

When Reference is set to Groups all groups that are Applicable to this filter are considered a background reference. For each such group the filter will calculate a distance between the average of that group and each input pixel. A pixel is considered foreground if the comparison between any distance calculated thusly against the specified threshold evaluates to true.

Typically the user will specify one single group that will be considered as the background reference.

Scene

When Reference is set to Scene the filter attempts to determine the background automatically from the contents of the scene. This assumes that the first few rows (i.e. the y direction of the image) will only contain the actual background and not any objects.

Here the filter will assume that the first rows only contain background pixels (and not attempt to classify them at all), and average the data of each pixel in those rows to the background reference that will then be used for subsequent rows.

The Scene Reference Height parameter specifies how many rows the filter should add to the average that will be used for the background reference before starting the classification process. A higher number here will improve the quality of the average (because averaging more pixels will reduce the effect of noise), but the user should take care to only select as many rows as they are sure that the scene will contain only background pixels.

In Individual Column Mode the basic logic is the same, but each individual column is treated separately, i.e. it will have its own distinct background reference. In that case it is recommended to increase the Scene Reference Height parameter, because when using a reference height of 1, each background reference will consist of only a single pixel (that of the first row), causing the logic of the filter to be much more susceptible to noise.

Exclusion Zones

When Column Mode is set to Exclusion Zones the filter changes how the classification is performed. In this mode the filter will also assume that the first Scene Reference Height rows will only consist of background. (This is the case even if Mode is Average and/or Reference is Groups.) It will then attempt to classify the pixels those first rows to determine which columns contain actual background in those first rows. (If Reference is Scene the top-left most pixel is always considered to be background so that the filter always has an initial reference to use.)

After those first rows the filter will then not perform any classification for those columns. Instead, for each row it processes, it will first classify the pixels corresponding to all other columns in that row, and then use the nearest classification result as the value for the remaining columns.

This mode is useful when used in conjunction with a line camera that overlooks a scene where the background of the scene is interrupted in places by e.g. a linear motion axis or a conveyor belt. In that case the filter can automatically determine which columns of the scene will look at those interruptions of the backgronud when there is no object in view of the camera, and not erroneously consider the pixels of those columns to always be foreground pixels, but instead use the neighboring pixels to make that determination.