Visual Information Fidelity

A Gaussian scale mixture (GSM) is used to statistically model the wavelet coefficients of a steerable pyramid decomposition of an image.[5] The model is described below for a given subband of the multi-scale multi-orientation decomposition and can be extended to other subbands similarly. Let the wavelet coefficients in a given subband be ${\displaystyle {\mathcal {C}}=\{{\bar {C}}_{i}:i\in {\mathcal {I}}\}}$ where ${\displaystyle {\mathcal {I}}}$ denotes the set of spatial indices across the subband and each ${\displaystyle {\bar {C}}_{i}}$ is an ${\displaystyle M}$ dimensional vector. The subband is partitioned into non-overlapping blocks of ${\displaystyle M}$ coefficients each, where each block corresponds to ${\displaystyle {\bar {C}}_{i}}$. According to the GSM model,

$${\displaystyle {\mathcal {C}}={\mathcal {S}}\cdot {\mathcal {U}}=\{S_{i}{\bar {U}}_{i}:i\in {\mathcal {I}}\},}$$

where ${\displaystyle S_{i}}$ is a positive scalar and ${\displaystyle {\bar {U}}_{i}}$ is a Gaussian vector with mean zero and co-variance ${\displaystyle \mathbf {C} _{U}}$. Further the non-overlapping blocks are assumed to be independent of each other and that the random field ${\displaystyle {\mathcal {S}}}$ is independent of ${\displaystyle {\mathcal {U}}}$.

The distortion process is modeled using a combination of signal attenuation and additive noise in the wavelet domain. Mathematically, if ${\displaystyle {\mathcal {D}}=\{{\bar {D}}_{i}:i\in {\mathcal {I}}\}}$ denotes the random field from a given subband of the distorted image, ${\displaystyle {\mathcal {G}}=\{g_{i}:i\in {\mathcal {I}}\}}$ is a deterministic scalar field and ${\displaystyle {\mathcal {V}}=\{{\bar {V}}_{i}:i\in {\mathcal {I}}\}}$, where ${\displaystyle {\bar {V}}_{i}}$ is a zero mean Gaussian vector with co-variance ${\displaystyle \mathbf {C} _{V}=\sigma _{v}^{2}\mathbf {I} }$, then

$${\displaystyle {\mathcal {D}}={\mathcal {G}}{\mathcal {C}}+{\mathcal {V}}.}$$

Further, ${\displaystyle {\mathcal {V}}}$ is modeled to be independent of ${\displaystyle {\mathcal {S}}}$ and ${\displaystyle {\mathcal {U}}}$.

The duality of HVS models and NSS implies that several aspects of the HVS have already been accounted for in the source model. Here, the HVS is additionally modeled based on the hypothesis that the uncertainty in the perception of visual signals limits the amount of information that can be extracted from the source and distorted image. This source of uncertainty can be modeled as visual noise in the HVS model. In particular, the HVS noise in a given subband of the wavelet decomposition is modeled as additive white Gaussian noise. Let ${\displaystyle {\mathcal {N}}=\{{\bar {N}}_{i}:i\in {\mathcal {I}}\}}$ and ${\displaystyle {\mathcal {N}}'=\{{\bar {N}}_{i}':i\in {\mathcal {I}}\}}$ be random fields, where ${\displaystyle {\bar {N}}_{i}}$ and ${\displaystyle {\bar {N}}_{i}'}$ are zero mean Gaussian vectors with co-variance ${\displaystyle \mathbf {C} _{N}}$ and ${\displaystyle \mathbf {C} _{N}'}$. Further, let ${\displaystyle {\mathcal {E}}}$ and ${\displaystyle {\mathcal {F}}}$ denote the visual signal at the output of the HVS. Mathematically, we have ${\displaystyle {\mathcal {E}}={\mathcal {C}}+{\mathcal {N}}}$ and ${\displaystyle {\mathcal {F}}={\mathcal {D}}+{\mathcal {N}}'}$. Note that ${\displaystyle {\mathcal {N}}}$ and ${\displaystyle {\mathcal {N}}'}$ are random fields that are independent of ${\displaystyle {\mathcal {S}}}$, ${\displaystyle {\mathcal {U}}}$ and ${\displaystyle {\mathcal {V}}}$.