Computer stereo vision

The conventional structured-light vision (SLV) employs a structured light or laser, and finds projector-camera correspondences.[2][3]

The conventional active stereo vision (ASV) employs a structured light or laser, however, the stereo matching is performed only for camera-camera correspondences, in the same way as the passive stereo vision.

There is a hybrid technique, which utilizes both camera-camera and projector-camera correspondences.[4]

Where the image planes are not co-planar image rectification is required to adjust the images as if they were co-planar. This may be achieved by a linear transformation.

The images may also need rectification to make each image equivalent to the image taken from a pinhole camera projecting to a flat plane.

Smoothness is a measure of how similar colors that are close together are. There is an assumption that objects are more likely to be colored with a small number of colors. So if we detect two pixels with the same color they most likely belong to the same object.

The method described above for evaluating smoothness is based on information theory, and an assumption that the influence of the color of a voxel influences the color of nearby voxels according to the normal distribution on the distance between points. The model is based on approximate assumptions about the world.

Another method based on prior assumptions of smoothness is auto-correlation.

Smoothness is a property of the world. It is not inherently a property of an image. For example, an image constructed of random dots would have no smoothness, and inferences about neighboring points would be useless.

Theoretically smoothness, along with other properties of the world should be learnt. This appears to be what the human vision system does.

The normal distribution is

${\displaystyle P(x,\mu ,\sigma )={\frac {1}{\sigma {\sqrt {2\pi }}}}e^{-{\frac {(x-\mu )^{2}}{2\sigma ^{2}}}}}$

Probability is related to information content described by message length L,

${\displaystyle P(x)=2^{-L(x)}}$

${\displaystyle L(x)=-\log _{2}{P(x)}}$

so,

${\displaystyle L(x,\mu ,\sigma )=\log _{2}(\sigma {\sqrt {2\pi }})+{\frac {(x-\mu )^{2}}{2\sigma ^{2}}}\log _{2}e}$

For the purposes of comparing stereoscopic images, only the relative message length matters. Based on this, the information measure I, called the Sum of Squares of Differences (SSD) is,

${\displaystyle I(x,\mu ,\sigma )={\frac {(x-\mu )^{2}}{\sigma ^{2}}}}$

where,

${\displaystyle L(x,\mu ,\sigma )=\log _{2}(\sigma {\sqrt {2\pi }})+I(x,\mu ,\sigma ){\frac {\log _{2}e}{2}}}$

Because of the cost in processing time of squaring numbers in SSD, many implementations use Sum of Absolute Difference (SAD) as the basis for computing the information measure. Other methods use normalized cross correlation (NCC).

The least squares measure may be used to measure the information content of the stereoscopic images,[7] given depths at each point ${\displaystyle z(x,y)}$. Firstly the information needed to express one image in terms of the other is derived. This is called ${\displaystyle I_{m}}$.

A color difference function should be used to fairly measure the difference between colors. The color difference function is written cd in the following. The measure of the information needed to record the color matching between the two images is,

${\displaystyle I_{m}(z_{1},z_{2})={\frac {1}{\sigma _{m}^{2}}}\sum _{x,y}\operatorname {cd} (\operatorname {color} _{1}(x,y+{\frac {k}{z_{1}(x,y)}}),\operatorname {color} _{2}(x,y))^{2}}$

An assumption is made about the smoothness of the image. Assume that two pixels are more likely to be the same color, the closer the voxels they represent are. This measure is intended to favor colors that are similar being grouped at the same depth. For example, if an object in front occludes an area of sky behind, the measure of smoothness favors the blue pixels all being grouped together at the same depth.

The total measure of smoothness uses the distance between voxels as an estimate of the expected standard deviation of the color difference,

${\displaystyle I_{s}(z_{1},z_{2})={\frac {1}{2\sigma _{h}^{2}}}\sum _{i:\{1,2\}}\sum _{x_{1},y_{1}}\sum _{x_{2},y_{2}}{\frac {\operatorname {cd} (\operatorname {color} _{i}(x_{1},y_{1}),\operatorname {color} _{i}(x_{2},y_{2}))^{2}}{(x_{1}-x_{2})^{2}+(y_{1}-y_{2})^{2}+(z_{i}(x_{1},y_{1})-z_{i}(x_{2},y_{2}))^{2}}}}$

The total information content is then the sum,

${\displaystyle I_{t}(z_{1},z_{2})=I_{m}(z_{1},z_{2})+I_{s}(z_{1},z_{2})}$

The z component of each pixel must be chosen to give the minimum value for the information content. This will give the most likely depths at each pixel. The minimum total information measure is,

${\displaystyle I_{\operatorname {min} }=\min {\{i:i=I_{t}(z_{1},z_{2})\}}\}}$

The depth functions for the left and right images are the pair,

${\displaystyle (z_{1},z_{2})\in \{(z_{1},z_{2}):I_{t}(z_{1},z_{2})=I_{\operatorname {min} }\}}$