Functions and classes described in this section are used to perform various linear or non-linear filtering operations on 2D images (represented as Mat() ‘s), that is, for each pixel location in the source image some its (normally rectangular) neighborhood is considered and used to compute the response. In case of a linear filter it is a weighted sum of pixel values, in case of morphological operations it is the minimum or maximum etc. The computed response is stored to the destination image at the same location . It means, that the output image will be of the same size as the input image. Normally, the functions supports multi-channel arrays, in which case every channel is processed independently, therefore the output image will also have the same number of channels as the input one.
Another common feature of the functions and classes described in this section is that, unlike simple arithmetic functions, they need to extrapolate values of some non-existing pixels. For example, if we want to smooth an image using a Gaussian filter, then during the processing of the left-most pixels in each row we need pixels to the left of them, i.e. outside of the image. We can let those pixels be the same as the left-most image pixels (i.e. use “replicated border” extrapolation method), or assume that all the non-existing pixels are zeros (“contant border” extrapolation method) etc. OpenCV let the user to specify the extrapolation method; see the function borderInterpolate() and discussion of borderType parameter in various functions below.
Base class for filters with single-column kernels
class BaseColumnFilter
{
public:
virtual ~BaseColumnFilter();
// To be overriden by the user.
//
// runs filtering operation on the set of rows,
// "dstcount + ksize - 1" rows on input,
// "dstcount" rows on output,
// each input and output row has "width" elements
// the filtered rows are written into "dst" buffer.
virtual void operator()(const uchar** src, uchar* dst, int dststep,
int dstcount, int width) = 0;
// resets the filter state (may be needed for IIR filters)
virtual void reset();
int ksize; // the aperture size
int anchor; // position of the anchor point,
// normally not used during the processing
};
The class BaseColumnFilter is the base class for filtering data using single-column kernels. The filtering does not have to be a linear operation. In general, it could be written as following:
where is the filtering function, but, as it is represented as a class, it can produce any side effects, memorize previously processed data etc. The class only defines the interface and is not used directly. Instead, there are several functions in OpenCV (and you can add more) that return pointers to the derived classes that implement specific filtering operations. Those pointers are then passed to FilterEngine() constructor. While the filtering operation interface uses uchar type, a particular implementation is not limited to 8-bit data.
See also: BaseRowFilter() , BaseFilter() , FilterEngine() ,
getColumnSumFilter() , getLinearColumnFilter() , getMorphologyColumnFilter()
Base class for 2D image filters
class BaseFilter
{
public:
virtual ~BaseFilter();
// To be overriden by the user.
//
// runs filtering operation on the set of rows,
// "dstcount + ksize.height - 1" rows on input,
// "dstcount" rows on output,
// each input row has "(width + ksize.width-1)*cn" elements
// each output row has "width*cn" elements.
// the filtered rows are written into "dst" buffer.
virtual void operator()(const uchar** src, uchar* dst, int dststep,
int dstcount, int width, int cn) = 0;
// resets the filter state (may be needed for IIR filters)
virtual void reset();
Size ksize;
Point anchor;
};
The class BaseFilter is the base class for filtering data using 2D kernels. The filtering does not have to be a linear operation. In general, it could be written as following:
where is the filtering function. The class only defines the interface and is not used directly. Instead, there are several functions in OpenCV (and you can add more) that return pointers to the derived classes that implement specific filtering operations. Those pointers are then passed to FilterEngine() constructor. While the filtering operation interface uses uchar type, a particular implementation is not limited to 8-bit data.
See also: BaseColumnFilter() , BaseRowFilter() , FilterEngine() ,
Base class for filters with single-row kernels
class BaseRowFilter
{
public:
virtual ~BaseRowFilter();
// To be overriden by the user.
//
// runs filtering operation on the single input row
// of "width" element, each element is has "cn" channels.
// the filtered row is written into "dst" buffer.
virtual void operator()(const uchar* src, uchar* dst,
int width, int cn) = 0;
int ksize, anchor;
};
The class BaseRowFilter is the base class for filtering data using single-row kernels. The filtering does not have to be a linear operation. In general, it could be written as following:
where is the filtering function. The class only defines the interface and is not used directly. Instead, there are several functions in OpenCV (and you can add more) that return pointers to the derived classes that implement specific filtering operations. Those pointers are then passed to FilterEngine() constructor. While the filtering operation interface uses uchar type, a particular implementation is not limited to 8-bit data.
See also: BaseColumnFilter() , Filter() , FilterEngine() ,
getLinearRowFilter() , getMorphologyRowFilter() , getRowSumFilter()
Generic image filtering class
class FilterEngine
{
public:
// empty constructor
FilterEngine();
// builds a 2D non-separable filter (!_filter2D.empty()) or
// a separable filter (!_rowFilter.empty() && !_columnFilter.empty())
// the input data type will be "srcType", the output data type will be "dstType",
// the intermediate data type is "bufType".
// _rowBorderType and _columnBorderType determine how the image
// will be extrapolated beyond the image boundaries.
// _borderValue is only used when _rowBorderType and/or _columnBorderType
// == cv::BORDER_CONSTANT
FilterEngine(const Ptr<BaseFilter>& _filter2D,
const Ptr<BaseRowFilter>& _rowFilter,
const Ptr<BaseColumnFilter>& _columnFilter,
int srcType, int dstType, int bufType,
int _rowBorderType=BORDER_REPLICATE,
int _columnBorderType=-1, // use _rowBorderType by default
const Scalar& _borderValue=Scalar());
virtual ~FilterEngine();
// separate function for the engine initialization
void init(const Ptr<BaseFilter>& _filter2D,
const Ptr<BaseRowFilter>& _rowFilter,
const Ptr<BaseColumnFilter>& _columnFilter,
int srcType, int dstType, int bufType,
int _rowBorderType=BORDER_REPLICATE, int _columnBorderType=-1,
const Scalar& _borderValue=Scalar());
// starts filtering of the ROI in an image of size "wholeSize".
// returns the starting y-position in the source image.
virtual int start(Size wholeSize, Rect roi, int maxBufRows=-1);
// alternative form of start that takes the image
// itself instead of "wholeSize". Set isolated to true to pretend that
// there are no real pixels outside of the ROI
// (so that the pixels will be extrapolated using the specified border modes)
virtual int start(const Mat& src, const Rect& srcRoi=Rect(0,0,-1,-1),
bool isolated=false, int maxBufRows=-1);
// processes the next portion of the source image,
// "srcCount" rows starting from "src" and
// stores the results to "dst".
// returns the number of produced rows
virtual int proceed(const uchar* src, int srcStep, int srcCount,
uchar* dst, int dstStep);
// higher-level function that processes the whole
// ROI or the whole image with a single call
virtual void apply( const Mat& src, Mat& dst,
const Rect& srcRoi=Rect(0,0,-1,-1),
Point dstOfs=Point(0,0),
bool isolated=false);
bool isSeparable() const { return filter2D.empty(); }
// how many rows from the input image are not yet processed
int remainingInputRows() const;
// how many output rows are not yet produced
int remainingOutputRows() const;
...
// the starting and the ending rows in the source image
int startY, endY;
// pointers to the filters
Ptr<BaseFilter> filter2D;
Ptr<BaseRowFilter> rowFilter;
Ptr<BaseColumnFilter> columnFilter;
};
The class FilterEngine can be used to apply an arbitrary filtering operation to an image. It contains all the necessary intermediate buffers, it computes extrapolated values of the “virtual” pixels outside of the image etc. Pointers to the initialized FilterEngine instances are returned by various create*Filter functions, see below, and they are used inside high-level functions such as filter2D() , erode() , dilate() etc, that is, the class is the workhorse in many of OpenCV filtering functions.
This class makes it easier (though, maybe not very easy yet) to combine filtering operations with other operations, such as color space conversions, thresholding, arithmetic operations, etc. By combining several operations together you can get much better performance because your data will stay in cache. For example, below is the implementation of Laplace operator for a floating-point images, which is a simplified implementation of Laplacian() :
void laplace_f(const Mat& src, Mat& dst)
{
CV_Assert( src.type() == CV_32F );
dst.create(src.size(), src.type());
// get the derivative and smooth kernels for d2I/dx2.
// for d2I/dy2 we could use the same kernels, just swapped
Mat kd, ks;
getSobelKernels( kd, ks, 2, 0, ksize, false, ktype );
// let's process 10 source rows at once
int DELTA = std::min(10, src.rows);
Ptr<FilterEngine> Fxx = createSeparableLinearFilter(src.type(),
dst.type(), kd, ks, Point(-1,-1), 0, borderType, borderType, Scalar() );
Ptr<FilterEngine> Fyy = createSeparableLinearFilter(src.type(),
dst.type(), ks, kd, Point(-1,-1), 0, borderType, borderType, Scalar() );
int y = Fxx->start(src), dsty = 0, dy = 0;
Fyy->start(src);
const uchar* sptr = src.data + y*src.step;
// allocate the buffers for the spatial image derivatives;
// the buffers need to have more than DELTA rows, because at the
// last iteration the output may take max(kd.rows-1,ks.rows-1)
// rows more than the input.
Mat Ixx( DELTA + kd.rows - 1, src.cols, dst.type() );
Mat Iyy( DELTA + kd.rows - 1, src.cols, dst.type() );
// inside the loop we always pass DELTA rows to the filter
// (note that the "proceed" method takes care of possibe overflow, since
// it was given the actual image height in the "start" method)
// on output we can get:
// * < DELTA rows (the initial buffer accumulation stage)
// * = DELTA rows (settled state in the middle)
// * > DELTA rows (then the input image is over, but we generate
// "virtual" rows using the border mode and filter them)
// this variable number of output rows is dy.
// dsty is the current output row.
// sptr is the pointer to the first input row in the portion to process
for( ; dsty < dst.rows; sptr += DELTA*src.step, dsty += dy )
{
Fxx->proceed( sptr, (int)src.step, DELTA, Ixx.data, (int)Ixx.step );
dy = Fyy->proceed( sptr, (int)src.step, DELTA, d2y.data, (int)Iyy.step );
if( dy > 0 )
{
Mat dstripe = dst.rowRange(dsty, dsty + dy);
add(Ixx.rowRange(0, dy), Iyy.rowRange(0, dy), dstripe);
}
}
}
If you do not need that much control of the filtering process, you can simply use the FilterEngine::apply method. Here is how the method is actually implemented:
void FilterEngine::apply(const Mat& src, Mat& dst,
const Rect& srcRoi, Point dstOfs, bool isolated)
{
// check matrix types
CV_Assert( src.type() == srcType && dst.type() == dstType );
// handle the "whole image" case
Rect _srcRoi = srcRoi;
if( _srcRoi == Rect(0,0,-1,-1) )
_srcRoi = Rect(0,0,src.cols,src.rows);
// check if the destination ROI is inside the dst.
// and FilterEngine::start will check if the source ROI is inside src.
CV_Assert( dstOfs.x >= 0 && dstOfs.y >= 0 &&
dstOfs.x + _srcRoi.width <= dst.cols &&
dstOfs.y + _srcRoi.height <= dst.rows );
// start filtering
int y = start(src, _srcRoi, isolated);
// process the whole ROI. Note that "endY - startY" is the total number
// of the source rows to process
// (including the possible rows outside of srcRoi but inside the source image)
proceed( src.data + y*src.step,
(int)src.step, endY - startY,
dst.data + dstOfs.y*dst.step +
dstOfs.x*dst.elemSize(), (int)dst.step );
}
Unlike the earlier versions of OpenCV, now the filtering operations fully support the notion of image ROI, that is, pixels outside of the ROI but inside the image can be used in the filtering operations. For example, you can take a ROI of a single pixel and filter it - that will be a filter response at that particular pixel (however, it’s possible to emulate the old behavior by passing isolated=false to FilterEngine::start or FilterEngine::apply ). You can pass the ROI explicitly to FilterEngine::apply , or construct a new matrix headers:
// compute dI/dx derivative at src(x,y)
// method 1:
// form a matrix header for a single value
float val1 = 0;
Mat dst1(1,1,CV_32F,&val1);
Ptr<FilterEngine> Fx = createDerivFilter(CV_32F, CV_32F,
1, 0, 3, BORDER_REFLECT_101);
Fx->apply(src, Rect(x,y,1,1), Point(), dst1);
// method 2:
// form a matrix header for a single value
float val2 = 0;
Mat dst2(1,1,CV_32F,&val2);
Mat pix_roi(src, Rect(x,y,1,1));
Sobel(pix_roi, dst2, dst2.type(), 1, 0, 3, 1, 0, BORDER_REFLECT_101);
printf("method1 =
Note on the data types. As it was mentioned in BaseFilter() description, the specific filters can process data of any type, despite that Base*Filter::operator() only takes uchar pointers and no information about the actual types. To make it all work, the following rules are used:
See also: BaseColumnFilter() , BaseFilter() , BaseRowFilter() , createBoxFilter() , createDerivFilter() , createGaussianFilter() , createLinearFilter() , createMorphologyFilter() , createSeparableLinearFilter()
Applies bilateral filter to the image
Parameters: |
|
---|
The function applies bilateral filtering to the input image, as described in http://www.dai.ed.ac.uk/CVonline/LOCAL_COPIES/MANDUCHI1/Bilateral_Filtering.html
Smoothes image using normalized box filter
Parameters: |
|
---|
The function smoothes the image using the kernel:
The call blur(src, dst, ksize, anchor, borderType) is equivalent to boxFilter(src, dst, src.type(), anchor, true, borderType) .
See also: boxFilter() , bilateralFilter() , GaussianBlur() , medianBlur() .
Computes source location of extrapolated pixel
Parameters: |
|
---|
The function computes and returns the coordinate of the donor pixel, corresponding to the specified extrapolated pixel when using the specified extrapolation border mode. For example, if we use BORDER_WRAP mode in the horizontal direction, BORDER_REFLECT_101 in the vertical direction and want to compute value of the “virtual” pixel Point(-5, 100) in a floating-point image img , it will be
float val = img.at<float>(borderInterpolate(100, img.rows, BORDER_REFLECT_101),
borderInterpolate(-5, img.cols, BORDER_WRAP));
Normally, the function is not called directly; it is used inside FilterEngine() and copyMakeBorder() to compute tables for quick extrapolation.
See also: FilterEngine() , copyMakeBorder()
Smoothes image using box filter
Parameters: |
|
---|
The function smoothes the image using the kernel:
where
Unnormalized box filter is useful for computing various integral characteristics over each pixel neighborhood, such as covariation matrices of image derivatives (used in dense optical flow algorithms, etc.). If you need to compute pixel sums over variable-size windows, use integral() .
See also: boxFilter() , bilateralFilter() , GaussianBlur() , medianBlur() , integral() .
Constructs Gaussian pyramid for an image
Parameters: |
|
---|
The function constructs a vector of images and builds the gaussian pyramid by recursively applying pyrDown() to the previously built pyramid layers, starting from dst[0]==src .
Forms a border around the image
Parameters: |
|
---|
The function copies the source image into the middle of the destination image. The areas to the left, to the right, above and below the copied source image will be filled with extrapolated pixels. This is not what FilterEngine() or based on it filtering functions do (they extrapolate pixels on-fly), but what other more complex functions, including your own, may do to simplify image boundary handling.
The function supports the mode when src is already in the middle of dst . In this case the function does not copy src itself, but simply constructs the border, e.g.:
// let border be the same in all directions
int border=2;
// constructs a larger image to fit both the image and the border
Mat gray_buf(rgb.rows + border*2, rgb.cols + border*2, rgb.depth());
// select the middle part of it w/o copying data
Mat gray(gray_canvas, Rect(border, border, rgb.cols, rgb.rows));
// convert image from RGB to grayscale
cvtColor(rgb, gray, CV_RGB2GRAY);
// form a border in-place
copyMakeBorder(gray, gray_buf, border, border,
border, border, BORDER_REPLICATE);
// now do some custom filtering ...
...
See also: borderInterpolate()
Returns box filter engine
Parameters: |
|
---|
The function is a convenience function that retrieves horizontal sum primitive filter with getRowSumFilter() , vertical sum filter with getColumnSumFilter() , constructs new FilterEngine() and passes both of the primitive filters there. The constructed filter engine can be used for image filtering with normalized or unnormalized box filter.
The function itself is used by blur() and boxFilter() .
See also: FilterEngine() , blur() , boxFilter() .
Returns engine for computing image derivatives
Parameters: |
|
---|
The function createDerivFilter() is a small convenience function that retrieves linear filter coefficients for computing image derivatives using getDerivKernels() and then creates a separable linear filter with createSeparableLinearFilter() . The function is used by Sobel() and Scharr() .
See also: createSeparableLinearFilter() , getDerivKernels() , Scharr() , Sobel() .
Returns engine for smoothing images with a Gaussian filter
Parameters: |
|
---|
The function createGaussianFilter() computes Gaussian kernel coefficients and then returns separable linear filter for that kernel. The function is used by GaussianBlur() . Note that while the function takes just one data type, both for input and output, you can pass by this limitation by calling getGaussianKernel() and then createSeparableFilter() directly.
See also: createSeparableLinearFilter() , getGaussianKernel() , GaussianBlur() .
Creates non-separable linear filter engine
Parameters: |
|
---|
The function returns pointer to 2D linear filter for the specified kernel, the source array type and the destination array type. The function is a higher-level function that calls getLinearFilter and passes the retrieved 2D filter to FilterEngine() constructor.
See also: createSeparableLinearFilter() , FilterEngine() , filter2D()
Creates engine for non-separable morphological operations
Parameters: |
|
---|
The functions construct primitive morphological filtering operations or a filter engine based on them. Normally it’s enough to use createMorphologyFilter() or even higher-level erode() , dilate() or morphologyEx() , Note, that createMorphologyFilter() analyses the structuring element shape and builds a separable morphological filter engine when the structuring element is square.
See also: erode() , dilate() , morphologyEx() , FilterEngine()
Creates engine for separable linear filter
Parameters: |
|
---|
The functions construct primitive separable linear filtering operations or a filter engine based on them. Normally it’s enough to use createSeparableLinearFilter() or even higher-level sepFilter2D() . The function createMorphologyFilter() is smart enough to figure out the symmetryType for each of the two kernels, the intermediate bufType , and, if the filtering can be done in integer arithmetics, the number of bits to encode the filter coefficients. If it does not work for you, it’s possible to call getLinearColumnFilter , getLinearRowFilter directly and then pass them to FilterEngine() constructor.
See also: sepFilter2D() , createLinearFilter() , FilterEngine() , getKernelType()
Dilates an image by using a specific structuring element.
Parameters: |
|
---|
The function dilates the source image using the specified structuring element that determines the shape of a pixel neighborhood over which the maximum is taken:
The function supports the in-place mode. Dilation can be applied several ( iterations ) times. In the case of multi-channel images each channel is processed independently.
See also: erode() , morphologyEx() , createMorphologyFilter()
Erodes an image by using a specific structuring element.
Parameters: |
|
---|
The function erodes the source image using the specified structuring element that determines the shape of a pixel neighborhood over which the minimum is taken:
The function supports the in-place mode. Erosion can be applied several ( iterations ) times. In the case of multi-channel images each channel is processed independently.
See also: dilate() , morphologyEx() , createMorphologyFilter()
Convolves an image with the kernel
Parameters: |
|
---|
The function applies an arbitrary linear filter to the image. In-place operation is supported. When the aperture is partially outside the image, the function interpolates outlier pixel values according to the specified border mode.
The function does actually computes correlation, not the convolution:
That is, the kernel is not mirrored around the anchor point. If you need a real convolution, flip the kernel using flip() and set the new anchor to (kernel.cols - anchor.x - 1, kernel.rows - anchor.y - 1) .
The function uses -based algorithm in case of sufficiently large kernels (~ ) and the direct algorithm (that uses the engine retrieved by createLinearFilter() ) for small kernels.
See also: sepFilter2D() , createLinearFilter() , dft() , matchTemplate()
Smoothes image using a Gaussian filter
Parameters: |
|
---|
The function convolves the source image with the specified Gaussian kernel. In-place filtering is supported.
See also: sepFilter2D() , filter2D() , blur() , boxFilter() , bilateralFilter() , medianBlur()
Returns filter coefficients for computing spatial image derivatives
Parameters: |
|
---|
The function computes and returns the filter coefficients for spatial image derivatives. When ksize=CV_SCHARR , the Scharr kernels are generated, see Scharr() . Otherwise, Sobel kernels are generated, see Sobel() . The filters are normally passed to sepFilter2D() or to createSeparableLinearFilter() .
Returns Gaussian filter coefficients
Parameters: |
|
---|
The function computes and returns the matrix of Gaussian filter coefficients:
where and is the scale factor chosen so that Two of such generated kernels can be passed to sepFilter2D() or to createSeparableLinearFilter() that will automatically detect that these are smoothing kernels and handle them accordingly. Also you may use the higher-level GaussianBlur() .
See also: sepFilter2D() , createSeparableLinearFilter() , getDerivKernels() , getStructuringElement() , GaussianBlur() .
Returns the kernel type
Parameters: |
|
---|
The function analyzes the kernel coefficients and returns the corresponding kernel type:
- KERNEL_GENERAL Generic kernel - when there is no any type of symmetry or other properties
- KERNEL_SYMMETRICAL The kernel is symmetrical: and the anchor is at the center
- KERNEL_ASYMMETRICAL The kernel is asymmetrical: and the anchor is at the center
- KERNEL_SMOOTH All the kernel elements are non-negative and sum to 1. E.g. the Gaussian kernel is both smooth kernel and symmetrical, so the function will return KERNEL_SMOOTH | KERNEL_SYMMETRICAL
- KERNEL_INTEGER Al the kernel coefficients are integer numbers. This flag can be combined with KERNEL_SYMMETRICAL or KERNEL_ASYMMETRICAL
Returns the structuring element of the specified size and shape for morphological operations
Parameters: |
|
---|
The function constructs and returns the structuring element that can be then passed to createMorphologyFilter() , erode() , dilate() or morphologyEx() . But also you can construct an arbitrary binary mask yourself and use it as the structuring element.
Smoothes image using median filter
Parameters: |
|
---|
The function smoothes image using the median filter with aperture. Each channel of a multi-channel image is processed independently. In-place operation is supported.
See also: bilateralFilter() , blur() , boxFilter() , GaussianBlur()
Performs advanced morphological transformations
Parameters: |
|
---|
The function can perform advanced morphological transformations using erosion and dilation as basic operations.
Opening:
Closing:
Morphological gradient:
“Top hat”:
“Black hat”:
Any of the operations can be done in-place.
See also: dilate() , erode() , createMorphologyFilter()
Calculates the Laplacian of an image
Parameters: |
|
---|
The function calculates the Laplacian of the source image by adding up the second x and y derivatives calculated using the Sobel operator:
This is done when ksize > 1 . When ksize == 1 , the Laplacian is computed by filtering the image with the following aperture:
Smoothes an image and downsamples it.
Parameters: |
|
---|
The function performs the downsampling step of the Gaussian pyramid construction. First it convolves the source image with the kernel:
and then downsamples the image by rejecting even rows and columns.
Upsamples an image and then smoothes it
Parameters: |
|
---|
The function performs the upsampling step of the Gaussian pyramid construction (it can actually be used to construct the Laplacian pyramid). First it upsamples the source image by injecting even zero rows and columns and then convolves the result with the same kernel as in pyrDown() , multiplied by 4.
Applies separable linear filter to an image
Parameters: |
|
---|
The function applies a separable linear filter to the image. That is, first, every row of src is filtered with 1D kernel rowKernel . Then, every column of the result is filtered with 1D kernel columnKernel and the final result shifted by delta is stored in dst .
See also: createSeparableLinearFilter() , filter2D() , Sobel() , GaussianBlur() , boxFilter() , blur() .
Calculates the first, second, third or mixed image derivatives using an extended Sobel operator
Parameters: |
|
---|
In all cases except 1, an separable kernel will be used to calculate the derivative. When , a or kernel will be used (i.e. no Gaussian smoothing is done). ksize = 1 can only be used for the first or the second x- or y- derivatives.
There is also the special value ksize = CV_SCHARR (-1) that corresponds to a Scharr filter that may give more accurate results than a Sobel. The Scharr aperture is
for the x-derivative or transposed for the y-derivative.
The function calculates the image derivative by convolving the image with the appropriate kernel:
The Sobel operators combine Gaussian smoothing and differentiation, so the result is more or less resistant to the noise. Most often, the function is called with ( xorder = 1, yorder = 0, ksize = 3) or ( xorder = 0, yorder = 1, ksize = 3) to calculate the first x- or y- image derivative. The first case corresponds to a kernel of:
and the second one corresponds to a kernel of:
See also: Scharr() , Lapacian() , sepFilter2D() , filter2D() , GaussianBlur()
Calculates the first x- or y- image derivative using Scharr operator
Parameters: |
|
---|
The function computes the first x- or y- spatial image derivative using Scharr operator. The call
is equivalent to