Project 2: Fun with Filters and Frequencies (GitHub repo)

Overview

This project consists of two parts. In the first part, we will implement and test various filters, such as Gaussian (blur) filtering, gradient filtering, and sharpening filtering. In the second part, we will explore and apply frequency domain techniques for image blending and fusion.

Part 1: Fun with Filters

1.1: Finite Difference Operator

In this section, finite difference filters (see figure below) will be used as edge detectors for an image. These filters will produce two images representing 'dI/dx' and 'dI/dy', where 'I' represents the image array. Afterward, the magnitude of the edge gradient will be computed as |gradient(I)| = sqrt((dI/dx)^2 + (dI/dy)^2). The resulting gradient image will then be binarized using a predefined threshold, and the output will represent the edges of the original image.

Noted that dI/dx, dI/dy image pixel values are rescaled/normlaized back to [0,255] range for visulization purpose, the image processing process are done using the orignal pixel values.

1.2: Derivative of Gaussian (DoG) Filter

In the previous section, we observed some noise in the background of the binarized edge image (notice the tiny white dots, especially on the grass). One way to address this is by applying a Gaussian filter G to blur or smooth the image before the edge detection operation. After applying the filter, most of the noise is removed in the final binarized edge image. However, as a side effect, the edges become smoother and slightly wider.

The mathematical expression for dI/dx_blur is (dI/dx)_blur = Dx * (G * I), where "*" denotes the convolution operator. Due to the associativity of convolution, (dI/dx)_blur = (Dx * G) * I. We can create a new filter called the Derivative of Gaussian (DoG) filter, which is defined as Dx * G. Applying this filter to the original image will produce the same result as applying the Gaussian filter first and then performing the edge detection process, as shown in the figure below.

Part 2: Fun with Frequencies

2.1: Image "Sharpening"

The Gaussian filter is a low-pass filter, meaning it only retains the low frequencies of the image. Therefore, we can obtain the high frequencies by subtracting the blurred image from the original image. An image often appears "sharper" when it contains high-frequency components. To sharpen the image, we can amplify the high frequencies and add them back to the original image. The mathematical expression is: I_sharp = I + a(I - (G * I)) = I + aI - aI * G = (1 + a)I - aI * G = I * ((1 + a) * e - a * G). We can create a new "sharpening" filter (also called an unsharp mask) which is defined as (1 + a) * e - a * G, where "a" is the gain representing how much we want to amplify the high frequencies, and "e" is the "unit impulse" (i.e., it only has a value of 1 at the center of the image and 0 elsewhere).

For evaluation, I selected a sharp image, blurred it, and then resharpened it, as shown in the figure below. As we can see, although the sharpening filter can reduce the blur, the image still does not appear as sharp as the original. This is because the information removed by the Gaussian filter is lost, and without that information, we cannot fully recover the image to its original state.

2.2: Hybrid Images

High frequencies tend to dominate human perception when viewed up close, while low frequencies dominate at a distance. Based on this idea, we can blend two images into one so that people see one image from a distance and the other when viewed up close. We can use a Gaussian filter to obtain the low-frequency component of an image and use an impulse filter minus the Gaussian filter (e - G) to obtain the high-frequency part, since I_high_freq = I - (G * I) = I * (e - G).

FFT Analysis

Below are the FFT analysis of the blending process.

We can see that after low-pass filtering, most of the high frequencies (far away from the center area) are suppressed.

We can see that after high-pass filtering, most of the low frequencies (close to the center area) are suppressed.

Below are the blended image and its FFT.

Blending Combination (Bouns)

Here we used only grayscale images to blend, but we can also explore different blending combinations (e.g., grayscale + color, color + color). Personally, I find that using grayscale for the low-frequency input image and color for the high-frequency input image produces better results. This may be because, at close range, humans perceive color well, and color can enhance the high-frequency components of the image. If we use color for the low-frequency image, it may draw more attention to the low-frequency part and weaken the high-frequency details when viewed up close.

More Blending Result

2.3: Gaussian and Laplacian Stacks

Gaussian stack is the collection of Gaussian-filtered images from its previous level, starting from the original image (unfiltered).

In Laplacian stack, the image at level i is obtained by subtracting the corresponding Gaussian stack layer i + 1 from Gaussian stack layer i. To facilitate reconstruction of the original image, I save the lowest resolution layer of the Gaussian stack at the end of the Laplacian stack.

"Oraple"

Using the Gaussian and Laplcian stack, we can replicate the "Oraple" presented by Burt and Adelson in their paper.

We have two image that wnat to blend: an apple and an orange.

We can construct the Gaussian stackan and the Laplacian stack.

Final result

"Apple Man"

"Two Face"