Structural similarity

The predecessor of SSIM was called Universal Quality Index (UQI), or Wang–Bovik Index, which was developed by Zhou Wang and Alan Bovik in 2001. This evolved, through their collaboration with Hamid Sheikh and Eero Simoncelli, into the current version of SSIM, which was published in April 2004 in the IEEE Transactions on Image Processing.[1] In addition to defining the SSIM quality index, the paper provides a general context for developing and evaluating perceptual quality measures, including connections to human visual neurobiology and perception, and direct validation of the index against human subject ratings.

The basic model was developed in the Laboratory for Image and Video Engineering (LIVE) at The University of Texas at Austin and further developed jointly with the Laboratory for Computational Vision (LCV) at New York University. Further variants of the model have been developed in the Image and Visual Computing Laboratory at University of Waterloo and have been commercially marketed.

SSIM subsequently found strong adoption in the image processing community. The 2004 SSIM paper has been cited over 20,000 times according to Google Scholar,[2] making it one of the highest cited papers in the image processing and video engineering fields. It was accorded the IEEE Signal Processing Society Best Paper Award for 2009.[3] It also received the IEEE Signal Processing Society Sustained Impact Award for 2016, indicative of a paper having an unusually high impact for at least 10 years following its publication.

The SSIM index is calculated on various windows of an image. The measure between two windows ${\displaystyle x}$ and ${\displaystyle y}$ of common size N×N is:[4]

$${\displaystyle {\hbox{SSIM}}(x,y)={\frac {(2\mu _{x}\mu _{y}+c_{1})(2\sigma _{xy}+c_{2})}{(\mu _{x}^{2}+\mu _{y}^{2}+c_{1})(\sigma _{x}^{2}+\sigma _{y}^{2}+c_{2})}}}$$

with:

• ${\displaystyle \scriptstyle \mu _{x}}$ the average of ${\displaystyle \scriptstyle x}$;
• ${\displaystyle \scriptstyle \mu _{y}}$ the average of ${\displaystyle \scriptstyle y}$;
• ${\displaystyle \scriptstyle \sigma _{x}^{2}}$ the variance of ${\displaystyle \scriptstyle x}$;
• ${\displaystyle \scriptstyle \sigma _{y}^{2}}$ the variance of ${\displaystyle \scriptstyle y}$;
• ${\displaystyle \scriptstyle \sigma _{xy}}$ the covariance of ${\displaystyle \scriptstyle x}$ and ${\displaystyle \scriptstyle y}$;
• ${\displaystyle \scriptstyle c_{1}=(k_{1}L)^{2}}$, ${\displaystyle \scriptstyle c_{2}=(k_{2}L)^{2}}$ two variables to stabilize the division with weak denominator;
• ${\displaystyle \scriptstyle L}$ the dynamic range of the pixel-values (typically this is ${\displaystyle \scriptstyle 2^{\#bits\ per\ pixel}-1}$);
• ${\displaystyle \scriptstyle k_{1}=0.01}$ and ${\displaystyle \scriptstyle k_{2}=0.03}$ by default.

Formula components

The SSIM formula is based on three comparison measurements between the samples of ${\displaystyle x}$ and ${\displaystyle y}$: luminance (${\displaystyle l}$), contrast (${\displaystyle c}$) and structure (${\displaystyle s}$). The individual comparison functions are:[4]

$${\displaystyle l(x,y)={\frac {2\mu _{x}\mu _{y}+c_{1}}{\mu _{x}^{2}+\mu _{y}^{2}+c_{1}}}}$$

$${\displaystyle c(x,y)={\frac {2\sigma _{x}\sigma _{y}+c_{2}}{\sigma _{x}^{2}+\sigma _{y}^{2}+c_{2}}}}$$

$${\displaystyle s(x,y)={\frac {\sigma _{xy}+c_{3}}{\sigma _{x}\sigma _{y}+c_{3}}}}$$

with, in addition to above definitions:

• ${\displaystyle c_{3}=c_{2}/2}$

SSIM is then a weighted combination of those comparative measures:

${\displaystyle {\text{SSIM}}(x,y)=\left[l(x,y)^{\alpha }\cdot c(x,y)^{\beta }\cdot s(x,y)^{\gamma }\right]}$

Setting the weights ${\displaystyle \alpha ,\beta ,\gamma }$ to 1, the formula can be reduced to the form shown above.

Structural Dissimilarity

Structural dissimilarity (DSSIM) may be derived from SSIM, though it does not constitute a distance function as the triangle inequality is not necessarily satisfied.

$${\displaystyle {\hbox{DSSIM}}(x,y)={\frac {1-{\hbox{SSIM}}(x,y)}{2}}}$$

Complex Wavelet SSIM

The complex wavelet transform variant of the SSIM (CW-SSIM) is designed to deal with issues of image scaling, translation and rotation. Instead of giving low scores to images with such conditions, the CW-SSIM takes advantage of the complex wavelet transform and therefore yields higher scores to said images. The CW-SSIM is defined as follows:

$${\displaystyle {\text{CW-SSIM}}(c_{x},c_{y})={\bigg (}{\frac {2\sum _{i=1}^{N}|c_{x,i}||c_{y,i}|+K}{\sum _{i=1}^{N}|c_{x,i}|^{2}+\sum _{i=1}^{N}|c_{y,i}|^{2}+K}}{\bigg )}{\bigg (}{\frac {2|\sum _{i=1}^{N}c_{x,i}c_{y,i}^{*}|+K}{2\sum _{i=1}^{N}|c_{x,i}c_{y,i}^{*}|+K}}{\bigg )}}$$

Where ${\displaystyle c_{x}}$ is the complex wavelet transform of the signal ${\displaystyle x}$ and ${\displaystyle c_{y}}$ is the complex wavelet transform for the signal ${\displaystyle y}$. Additionally, ${\displaystyle K}$ is a small positive number used for the purposes of function stability. Ideally, it should be zero. Like the SSIM, the CW-SSIM has a maximum value of 1. The maximum value of 1 indicates that the two signals are perfectly structurally similar while a value of 0 indicates no structural similarity.[12]

SSIM has applications in a variety of different problems. Some examples are:

• Image Compression: In lossy image compression, information is deliberately discarded to decrease the storage space of images and video. The MSE is typically used in such compression schemes. According to its authors, using SSIM instead of MSE is suggested to produce better results for the decompressed images.[12]
• Image Restoration: Image restoration focuses on solving the problem ${\displaystyle y=h*x+n}$ where ${\displaystyle y}$ is the blurry image that should be restored, ${\displaystyle h}$ is the blur kernel, ${\displaystyle n}$ is the additive noise and ${\displaystyle x}$ is the original image we wish to recover. The traditional filter which is used to solve this problem is the Wiener Filter. However, the Wiener filter design is based on the MSE. Using an SSIM variant, specifically Stat-SSIM, is claimed to produce better visual results, according to the algorithm's authors.[12]
• Pattern Recognition: Since SSIM mimics aspects of human perception, it could be used for recognizing patterns. When faced with issues like image scaling, translation and rotation, the algorithm's authors claim that it is better to use CW-SSIM,[15] which is insensitive to these variations and may be directly applied by template matching without using any training sample. Since data-driven pattern recognition approaches may produce better performance when a large amount of data is available for training, the authors suggest using CW-SSIM in data-driven approaches.[15]

Due to its popularity, SSIM is often compared to other metrics, including more simple metrics such as MSE and PSNR, and other perceptual image and video quality metrics. SSIM has been repeatedly shown to significantly outperform MSE and its derivates in accuracy, including research by its own authors and others.[7][16][17][18][19][20]

A paper by Dosselmann and Yang claims that the performance of SSIM is “much closer to that of the MSE” than usually assumed. While they do not dispute the advantage of SSIM over MSE, they state an analytical and functional dependency between the two metrics.[8] According to their research, SSIM has been found to correlate as well as MSE-based methods on subjective databases other than the databases from SSIM's creators. As an example, they cite Reibman and Poole, who found that MSE outperformed SSIM on a database containing packet-loss–impaired video.[21] In another paper, an analytical link between PSNR and SSIM was identified.[22]

• Mean squared error
• Peak signal-to-noise ratio
• Video quality

• Home page
• Rust Implementation
• C/C++ Implementation
• DSSIM C++ Implementation
• Chris Lomont's C# Implementation
• qpsnr implementation (multi threaded C++)
• Implementation in VQMT software
• Implementation in Python