Oversampled binary image sensor

Fig.1 The imaging model. The simplified architecture of a diffraction-limited imaging system. Incident light field ${\displaystyle \lambda _{0}(x)}$ passes through an optical lens, which acts like a linear system with a diffraction-limited point spread function (PSF). The result is a smoothed light field ${\displaystyle \lambda (x)}$, which is subsequently captured by the image sensor.

Consider a simplified camera model shown in Fig.1. The ${\displaystyle \lambda _{0}(x)}$ is the incoming light intensity field. By assuming that light intensities remain constant within a short exposure period, the field can be modeled as only a function of the spatial variable ${\displaystyle x}$. After passing through the optical system, the original light field ${\displaystyle \lambda _{0}(x)}$ gets filtered by the lens, which acts like a linear system with a given impulse response. Due to imperfections (e.g., aberrations) in the lens, the impulse response, a.k.a. the point spread function (PSF) of the optical system, cannot be a Dirac delta, thus, imposing a limit on the resolution of the observable light field. However, a more fundamental physical limit is due to light diffraction.[9] As a result, even if the lens is ideal, the PSF is still unavoidably a small blurry spot. In optics, such diffraction-limited spot is often called the Airy disk,[9] whose radius ${\displaystyle R_{a}}$ can be computed as

${\displaystyle R_{a}=1.22\,wf,}$

where ${\displaystyle w}$ is the wavelength of the light and ${\displaystyle f}$ is the F-number of the optical system. Due to the lowpass (smoothing) nature of the PSF, the resulting ${\displaystyle \lambda (x)}$ has a finite spatial-resolution, i.e., it has a finite number of degrees of freedom per unit space.

Fig.2 The model of the binary image sensor. The pixels (shown as "buckets") collect photons, the numbers of which are compared against a quantization threshold q. In the figure, we illustrate the case when q = 2. The pixel outputs are binary: ${\displaystyle b_{m}=1}$ (i.e., white pixels) if there are at least two photons received by the pixel; otherwise, ${\displaystyle b_{m}=0}$ (i.e., gray pixels).

Fig.2 illustrates the binary sensor model. The ${\displaystyle s_{m}}$ denote the exposure values accumulated by the sensor pixels. Depending on the local values of ${\displaystyle s_{m}}$, each pixel (depicted as "buckets" in the figure) collects a different number of photons hitting on its surface. ${\displaystyle y_{m}}$ is the number of photons impinging on the surface of the ${\displaystyle m}$th pixel during an exposure period. The relation between ${\displaystyle s_{m}}$ and the photon count ${\displaystyle y_{m}}$ is stochastic. More specifically, ${\displaystyle y_{m}}$ can be modeled as realizations of a Poisson random variable, whose intensity parameter is equal to ${\displaystyle s_{m}}$,

As a photosensitive device, each pixel in the image sensor converts photons to electrical signals, whose amplitude is proportional to the number of photons impinging on that pixel. In a conventional sensor design, the analog electrical signals are then quantized by an A/D converter into 8 to 14 bits (usually the more bits the better). But in the binary sensor, the quantizer is 1 bit. In Fig.2, ${\displaystyle b_{m}}$ is the quantized output of the ${\displaystyle m}$th pixel. Since the photon counts ${\displaystyle y_{m}}$ are drawn from random variables, so are the binary sensor output ${\displaystyle b_{m}}$.

If it is allowed to have temporal oversampling, i.e.,taking multiple consecutive and independent frames without changing the total exposure time ${\displaystyle \tau }$, the performance of the binary sensor is equivalent to the sensor with same number of spatial oversampling under certain condition.[2] It means that people can make trade off between spatial oversampling and temporal oversampling. This is quite important, since technology usually gives limitation on the size of the pixels and the exposure time.