Noise [part 2]

Article by James Zjalic (Verden Forensics) and Henk-Jan Lamfers (Foclar)

Introduction

In our previous blog article we discussed the causes and some possible solutions to noise in digital imagery. In this follow up article, alternative solutions for specific types of noise are presented.

Lowpass frequency and Wiener noise

A discussed within our previous article on blur, an image can be represented by its power spectrum. In this spectrum, the image signal is decomposed into a range of high and low frequencies, as demonstrated in Figure 1 a-c. The low frequencies correspond to the slowly varying local average, whilst the high frequencies correspond to both the small details and the inherent noise signal. By suppressing the high frequencies relative to the low ones, the noise signal is attenuated, but at the expense of image detail [1] page 240.

The Wiener noise filter [1] page 341, is a very specific implementation of the Wiener deblur filter. In this filter, the blur is assumed to be absent. So, no blur correction is applied, only the noise compensation. Different implementations of the lowpass frequency filters are available, e.g. Ideal, Gaussian and Butterworth. The implementations differ in how the separation/transition between low and high frequencies are handled (see Figure 2).

Denoise

The denoise filter is a noise suppression algorithm that considers the location of contours within an image. A contour is a line that connects points in an image with equal intensity (see Figure 3). The variation of intensity along a contour can therefore be assumed to be caused by noise. Averaging these contour intensity values (see Figure 3 line A) suppresses the noise, but as averaging the perpendicular intensity variations (see Figure 3 line B) will seriously blur the details in the image, it should be avoided. The denoise filter carefully considers the location of both flat intensity regions and the image contours, making it possible to reduce the noise level in the image without causing a loss of detail.

Median filter

The median filter ([2] page 108 and [3] page 130), is a noise reduction filter best applied as a solution for impulse noise. For every input pixel, the histogram of a local environment is calculated, and the resulting output pixel is the median value of the local histogram (see Figure 4), thus eliminating outlier pixel values. Besides a standard implementation of the median filter, Impress also offers the so-called band implementation (see Figure 5). This implementation has an extra parameter entitled ‘Threshold’ that is used to distinguish outlier pixels from ‘normal’ pixels. The filter only replaces the outlier pixels by the local histogram median value, leaving the normal pixels untouched.

Bilateral filter

The bilateral filter [2] page 110, is a variant of the lowpass filter. The resultant pixel gets the local weighted average of a small rectangle. However, the weights of the contributing pixel values are not only determined by the distance from the result pixel location (special weight): the larger the deviation of a pixel value from the local average value, the smaller the signal weight will be for that pixel. The total weight for a contributing pixel is the product of signal and special weight (see Figure 6). This approach reduces the undesirable influence of outlier pixels on the local averaging process.

Frame averaging

In cases in which noise reduction is required for a video sequence, it can be accomplished by so-called frame averaging [4] page 32. This is premised on the assumption that the scene is stationary, and thus the only difference between the pixel values from subsequent video frames within the same image coordinates is the temporal fluctuating noise signal. By determining the temporal average for each pixel location, a single image with suppressed noise signal can be constructed (see Figure 7).

It should be noted that if the frame averaging is applied to sequences with non-stationary scenes, the result will be a single image with low noise levels that is subject to a degree of blur due to the transitions between the scenes. This blur resembles a motion blur which can be present in images with a long shutter speed time. If the scene is only static for a certain period, preselection of only the stationary part should be used to get a good, blur free, result. If you want to remove the noise from a non-stationary region of interest, one option is to pre-process the imagery using image stabilization to fix the region to a single location for a certain time interval. Once this has been performed, the frames within the stationary region can be time-averaged. The frames outside the region might exhibit non-stationary behaviour that leads to blur, but these are of little consequence as they are outside of the required region.

Temporal smoothing

When noise exists within a video, the frame average filter only offers a solution in cases where h which the desired region is stationary. Where this is not the case, the so-called temporal smoothing filter [4] page 228 might be useful. Using the box filter from part 1, the average pixel values are calculated from the local neighbourhood of a pixel location in a process that can be described as spatial smoothing. The average pixel value for frame averaging is then calculated for all frames in a video. In contrast, the average pixel value in temporal smoothing is determined over a limited number of subsequent frames (see Figure 8). The weights of the pixel values of the subsequent frames can be constant for a certain interval (box function). Another implementation of this temporal smoothing has weights that become smaller as you go back further in time. The most recent frames contribute the most. A third implementation of temporal smoothing uses the temporal median value over a time range, eliminating outlier pixel values.

Conclusion

As previously stated, noise reduction is always a balance between the suppression of unwanted noise and the preservation of image details. Special situations (such as impulse noise) can be treated with special filters (such as a median filter). Video sequences offer extra options for the reduction of noise, e.g. frame averaging and temporal smoothing, for stationary and non-stationary scenes, respectively.

References

[1] William K. Pratt Digital Image Processing, Wiley 2007. 
[2] R. Szeliski Computer Vision, Algorithms and Applications, Springer 2011. 
[3] Sonka M. Hlavac V. Boyle R. Image Processing, Analysis and Machine Vision, Brooks/Cole Publishing Company 1999. 
[4] Alan C. Bovik, Handbook of Image and Video Processing Academic Press 2010.