Image derivatives

The derivative kernels, known as the Sobel operator are defined as follows, for the ${\displaystyle u}$ and ${\displaystyle v}$ directions respectively:

${\displaystyle p'_{u}={\begin{bmatrix}+1&+2&+1\\0&0&0\\-1&-2&-1\end{bmatrix}}*\mathbf {G} \quad {\mbox{and}}\quad p'_{v}={\begin{bmatrix}+1&0&-1\\+2&0&-2\\+1&0&-1\end{bmatrix}}*\mathbf {G} }$

where ${\displaystyle *}$ here denotes the 2-dimensional convolution operation.

This operator is separable and can be decomposed as the products of an interpolation and a differentiation kernel, so that, ${\displaystyle p'_{v}}$, for an example can be written as

${\displaystyle {\begin{bmatrix}+1&0&-1\\+2&0&-2\\+1&0&-1\end{bmatrix}}={\begin{bmatrix}1\\2\\1\end{bmatrix}}{\begin{bmatrix}+1&0&-1\end{bmatrix}}}$

Farid and Simoncelli[7][8] propose to use a pair of kernels, one for interpolation and another for differentiation (compare to Sobel above). These kernels, of fixed sizes 5 x 5 and 7 x 7, are optimized so that the Fourier transform approximates their correct derivative relationship.

In Matlab code the so called 5-tap filter is

k  = [0.030320  0.249724  0.439911  0.249724  0.030320];
d  = [0.104550  0.292315  0.000000 -0.292315 -0.104550];
d2 = [0.232905  0.002668 -0.471147  0.002668  0.232905];

And the 7-tap filter is

k  = [ 0.004711  0.069321  0.245410  0.361117  0.245410  0.069321  0.004711];
d  = [ 0.018708  0.125376  0.193091  0.000000 -0.193091 -0.125376 -0.018708];
d2 = [ 0.055336  0.137778 -0.056554 -0.273118 -0.056554  0.137778  0.055336];

As an example the first order derivatives can be computed in the following using Matlab in order to perform the convolution

 
Iu = conv2(d, k, im, 'same');  % derivative vertically (wrt Y)
Iv = conv2(k, d, im, 'same');  % derivative horizontally (wrt X)

It is noted that Farid and Simoncelli have derived first derivative coefficients which are more accurate compared to the ones provided above. However, the latter are consistent with the second derivative interpolator and, therefore, are better to use if both the first and second derivatives are sought. In the opposite case, when only the first derivative is desired, the optimal first derivative coefficients should be employed; more details can be found in their paper.

Derivative filters based on arbitrary cubic splines was presented by Hast.[9] He showed how both first and second order derivatives can be computed more correctly using cubic or trigonometric splines. Efficient derivative filters need to be of odd length so that the derivative is computed for the central pixel. However, any cubic filter is fitted over 4 sample points, giving a centre that falls between pixels. This is solved by a double filtering approach giving filters of size 7 x 7. The idea is to first filter by interpolation so that the interpolated value between pixels are obtained, whereafter the procedure is repeated using a derivative filters, where the centre value now falls on pixel centres. This can easily be proved by the associative law for convolution

${\displaystyle p'_{u}=\mathbf {d} \ast (\mathbf {k} \ast G)=(\mathbf {d} \ast \mathbf {k} )\ast G}$

Therefore the convolution kernel for computing the derivative ${\displaystyle \mathbf {k_{d}} }$ using an interpolating kernel ${\displaystyle \mathbf {k} }$ and a derivative kernel ${\displaystyle \mathbf {d} }$ becomes

${\displaystyle \mathbf {k_{d}} =\mathbf {d} \ast \mathbf {k} }$

Also keep in mind that convolution is commutative, so that the order of the two kernels does not matter and it is also possible to insert a second order derivative as well as a first order derivative kernel. These kernels are derived from the fact that any spline surface can be fitted over a square pixel region, compare to Bezier surfaces. Hast proves that such a surface can be performed as a separable convolution

${\displaystyle p(u,v)=\mathbf {u} ^{T}MGM^{T}\mathbf {v} =M^{T}\mathbf {u} \otimes \mathbf {v} ^{T}M\ast G}$

where ${\displaystyle M}$ is the spline basis matrix, ${\displaystyle \mathbf {u} }$ and ${\displaystyle \mathbf {v} }$ are vectors containing the variables ${\displaystyle u}$ and ${\displaystyle v}$, such as

${\displaystyle \mathbf {u} =[u^{3},u^{2},u,1]^{T}}$

${\displaystyle \mathbf {v} =[v^{3},v^{2},v,1]^{T}}$

The convolution kernels can now be set to

${\displaystyle \mathbf {k} =\mathbf {u} ^{T}M=(M^{T}\mathbf {v} )^{T}}$

${\displaystyle \mathbf {d} ={\frac {\partial \mathbf {u} ^{T}}{\partial u}}M=\left(M^{T}{\frac {\partial \mathbf {v} }{\partial v}}\right)^{T}}$

${\displaystyle \mathbf {d^{2}} ={\frac {\partial ^{2}\mathbf {u} ^{T}}{\partial u^{2}}}M=\left(M^{T}{\frac {\partial ^{2}\mathbf {v} }{\partial v^{2}}}\right)^{T}}$

The first order derivatives at the central pixel are hence computed as

${\displaystyle D_{u}={\frac {\partial \mathbf {u} ^{T}}{\partial u}}M\ast \mathbf {u} ^{T}MGM^{T}\mathbf {v} \ast M^{T}\mathbf {v} =\mathbf {d} \ast \mathbf {k} \otimes (\mathbf {k} \ast \mathbf {k} )^{T}\ast G}$

and

${\displaystyle D_{v}=\mathbf {u} ^{T}M\ast \mathbf {u} ^{T}MGM^{T}\mathbf {v} \ast M^{T}{\frac {\partial \mathbf {v} }{\partial v}}=\mathbf {k} \ast \mathbf {k} \otimes (\mathbf {d} \ast \mathbf {k} )^{T}\ast G}$

Likewise, with the second order derivative kernels are

${\displaystyle D_{u}^{2}={\frac {\partial ^{2}\mathbf {u} ^{T}}{\partial u^{2}}}M\ast \mathbf {u} ^{T}MGM^{T}\mathbf {v} \ast M^{T}\mathbf {v} =\mathbf {d^{2}} \ast \mathbf {k} \otimes (\mathbf {k} \ast \mathbf {k} )^{T}\ast G}$

and

${\displaystyle D_{v}^{2}=\mathbf {u} ^{T}M\ast \mathbf {u} ^{T}MGM^{T}\mathbf {v} \ast M^{T}{\frac {\partial ^{2}\mathbf {v} }{\partial v^{2}}}=\mathbf {k} \ast \mathbf {k} \otimes (\mathbf {d^{2}} \ast \mathbf {k} )^{T}\ast G}$

The cubic spline filter is evaluated in its centre ${\displaystyle u=v=0.5}$ and therefore

${\displaystyle \mathbf {u} =\mathbf {v} =[(0.5)^{3},(0.5)^{2},0.5,1]^{T}=[0.125,0.25,0.5,1]^{T}}$

Likewise the first order derivatives becomes

${\displaystyle {\frac {\partial \mathbf {u} (0.5)}{\partial u}}={\frac {\partial \mathbf {v} (0.5)}{\partial v}}=[3\cdot (0.5)^{2},2\cdot (0.5),1,0]^{T}=[0.75,1,1,0]^{T}}$

And in a similar manner the second order derivatives are

${\displaystyle {\frac {d^{2}\mathbf {u} (0.5)}{u^{2}}}={\frac {d^{2}\mathbf {v} (0.5)}{dv^{2}}}=[6\cdot (0.5),2,0,0]^{T}=[3,2,0,0]^{T}}$

Any cubic filter can be applied and used for computing the image derivates using the above equations, such as Bézier, Hermite or B-splines.

The example in below in Matlab use the Catmull-Rom spline to compute the derivatives

 
M = [1,-3,3,-1; -1,4,-5,2; 0,1,0,-1; 0,0,2,0] * 0.5;
u = [0.125;0.25;0.5;1];
up = [0.75;1;1;0];
d = up'*M;
k = u'*M;
Iu = conv2(conv(d,k), conv(k,k), im,'same');  % vertical derivative (wrt Y)
Iv = conv2(conv(k,k), conv(d,k), im,'same');  % horizontal derivative (wrt X)

Steerable filters can be used for computing derivatives[10] Moreover, Savitzky and Golay[11] propose a least-squares polynomial smoothing approach, which could be used for computing derivatives and Luo et al[12] discuss this approach in further detail. Scharr[13][14][15] shows how to create derivative filters by minimizing the error in the Fourier domain and Jähne et al[16] discuss in more detail the principles of filter design, including derivative filters.

• derivative5.m Farid and Simoncelli: 5-Tap 1st and 2nd discrete derivatives.
• derivative7.m Farid and Simoncelli: 7-Tap 1st and 2nd discrete derivatives
• kernel.m Hast: 1st and 2nd discrete derivatives for Cubic splines, Catmull-Rom splines, Bezier splines, B-Splines and Trigonometric splines.