Activity 9: Binary Operations

In image processing, especially if one is dealing with a region of interest (ROI) in an image, it is necessary to separate the ROI from the background. Thresholding is one way to effectively separate the two. This will reduce the image into a binary one. However, this requires an optimum threshold but this optimum threshold does not guarantee us a perfect separation of the ROI and the background. Instances that would sacrifice some of the ROI to zero out the background and vise-versa may be encountered. Hence, a threshold somewhere in between must be chosen that would leave some white pixels in the background and holes on the ROI.

In such cases (which happens more often), morphological operations must be performed for two basic reasons: (1) to remove the small white pixels on the background and holes on the ROI and (2) to separate touching blobs in the image. Blob separation is very important for accurate determination of cell sizes, etc. Here,we use the closing and opening operators to serve our purpose. Closing is just a dilation of the image followed by an erosion. Opening is the opposite.

Using the image below, we segment this into 256x256 sub-images. Threshold is applied to binarize each sub-image. Morphological operations is then performed per binarized sub-image. The area of each blob (in pixels) is calculated per sub-image. A histogram of blob areas is then obtained from all the sub-images. The thresholded is shown in Fig.2.

Figure 1. Grayscale image of ''cells''


Figure2. Thresholded sub-images in Fig.1

Before the process of opening and closing, choice of the structuring element (SE) is needed. In choosing, we have to take into account that the SE should not be large enough that upon dilation will not cause originally separated blobs to connect. As such, performing an erosion operation will not be able to separate them again. If, on the other hand, the SE is small enough, closure of relatively large holes will not be effective. Even repetitive closing would be futile. A balance between the two must be chosen.
Here, a 3x3 cross was chosen as the SE, since there are no large holes in the binary images obtained. One of the advantage of this SE is that this will not join blobs upon closing thus we can accurately determine the cell sizes.

Figure 3. Structuring element used for morphological operations

Using thus SE, we perform three opening to effectively remove the white pixels in the background and one closing to close the holes in the ROI. Then we use the bwlabel to label the blobs in each sub-image and calculate their areas in pixels. A figure below shows all the blobs obtained from all sub-images. Each blob are shaded differently.

Figure 4. Cleaned images from Fig.2.

From the areas computed, we plot the its histogram to approximate the sizes of the cells. In Fig.4, we can see that most blobs are composed only of one cell. Judging from this, we can pretty much predict that the peak of the histogram would occur at area of single-cell blobs. Below is the histogram obtained.

Figure 5. Histogram of areas in pixels

The peak corresponds to the approximate size of the cells. Examining Fig.5, we can infer that the cell size is approximately 550 pixels.

In this activity, I give myself a grade of 10 for doing the job well. I acknowledge Orly, Mark Jayson and Thirdy for useful discussions and laughter.

Activity 8 - Morphological Operations

In this activity, we were tasked to study dilation and erosion -- two of the morpholigical operation techniques applied in image processing. These operations (usually done in binary images) are used to enhance the image for better extraction of information. Morphological techniques have different effects on the image.
Let's take a look a dilation first. Using a structuring element; the dilation of an image is all the translation of the center of the element in the image such that their intersection is non-zero. Dilation, in effect, enlarges the image. The figure below demonstrates dilation of various images using different structuring elements.

Figure 1. Dilation using different structuring elements

Dilation of images can be done by hand. We just have to know the structuring element that would reshape the image. Before the simulation was done, the images as a result of dilation were first predicted. All my predictions were correct in comparison to the images obtained in Fig. 1.

Erosion, on the other hand, shrinks the image. One of the application of erosion is to remove unnecessary pixel on the image. Erosion takes all the translations of a structuring element such that all its pixels are on the image. Erosion can also be predicted by hand. The figure below demonstrates erosion of different images using different structuring elements.

Figure 2. Erosion of images using different structuring elements.

All my predictions were correct compared to images above.

In this activity, I give myself a grade of 10 for predicting properly the images as a result of dilation and erosion and also for properly creating the images above.

I thank Orly Tarun for uselful help.

Activity 7: Enhancement in the Frequency Domain

7A. Convolution Theorem

In this part, we were tasked to observe the changes in the modulus of the FT of the images as the size of the apertures are increased.

Figure1. 2-dot images and their FTs

Using two one-pixel dots, Fig.1(upper left), a grating-like FT was observed. As the radii of the dots increase, the FT becomes circle with decreasing radius located at the center of the two dots. this is expected since the FT is in inverse space.

Figure 2. square images and their FTs

Same was observed using squares. As the width of the squares increases, the width of their FTs decreases. Again, this is expected.

Also, for Gaussian dots, same was observed in Fig.1. As the variance of the Gaussian is increased - which results to an increase on the width of the Gaussian dots, their corresponding FTs is decreased.

Figure3. Guassian dot images at different variances and their FTs

We also take a look on how the FT of these Gaussian dot images differ from that of the inverted images (max becomes min; min becomes max). Here we plot in Fig.4 the images of the real part and imaginary part of the original and inverse Gaussian dots for comparison.

Figure 4. (upper row) images for the real part of original and inverse of 2 Gaussian dots; (lower row) images for the imaginary part of the original and inverse of 2 Gaussian dots.

We observe from the figure that only the real part changed upon inversion. The imaginary part remained the same.

7B. Fingerprints: Ridge Enhancements

In this part of the activity, we are to remove the blotches present in the image. Here we use the FT, we use a mask to remove the frequencies that result to the blotches and retain the information needed. We then take the inverse FT, we obtain a better image.

Figure 5. Removal of blotches present in the image in top left. A mask in bottom left was multiplied to the FT of the image (top right) to obtain a reconstructed image in bottom right.

As observed in the reconstructed image in Fig. 5 (bottom right), we essentially remove the blotches from the original image. Although the area surrounding the blotches darkened. This can be further improved if a better mask is used.

7C. Lunar Landing scanned Picture: Line removal

In this part of the activity, we were tasked to remove the vertical lines present in image below. Here, we use the Fourier transform to take information of the image in the frequency domain. We remove the frequencies that cause the vertical lines. From the previous exercise, we have learned that the frequencies of vertical lines are contained on the horizontal axis of the FT. Therefore, removal of frequencies lying on the horizontal axis removes the vertical lines.

Figure 6. Steps done (from upper left to lower right) to remove the vertical lines in the image.

From the FT in Fig.6(upper right), a mask was created Fig.6lower left) that will remove certain frequencies on the FT. The mask was then multiplied to the FT. Its FT was taken to recover the better image. In Fig.5, we see that the vertical lines were removed by removing the corresponding frequencies. Using FT, we were able to obtain a better image.

7D. Canvas Weave Modeling and Removal

In this part, we use again the technique demonstrated in 7C to remove the weave cross-hatches on the image in Fig.6(upper left). Since, we are dealing with cross-hatches (intertwined vertical and horizontal lines), we must take the frequencies on the horizontal and vertical axis of the FT domain. We might also consider that the cross-hatches are a result of the product of sinusoids (which has also been observed in Activity 3 Fig.5) and hence we also remove that constitute these sinusoids.

Figure 7. Steps of weave cross-hatches removal

Using the mask in Fig.7(lower left), a reconstructed image in Fig.7(lower right) was obtained. Here we see a decrease of cross-hatch visibility.

Figure 8. Weave pattern extracted from the image.

Furthermore, we can see in Fig. 8 a weave pattern extracted from the image. This was done by inverting the mask and multiplying it to the FT of the image and taking the inverse FT.

Although the technique produced promising results, our limit on reconstructing the image (or in other words our limit in creating better reconstructions) is dependent on the visibility of the peaks seen in the frequency domain. We can only block the frequencies that we see.

In this activity, I give myself a grade of 10 for doing a good job.

I thank Cindy Esporlas very much for helping me to create a Gaussian dot. I thank also Master and Thirdy for their help.

Activity 6 - Properties of the 2D Fourier Transform

6.A - Familiarization with FT of different 2D patterns

In this part of the activity, Fourier transforms (FT) of different apertures were investigated. Here, we familiarize on the patterns formed by this images upon applying FT.

Figure 1. (lower row) FTs of the images on the upper row.

The figure above shows the patterns obtained as a result of taking the FT of images of different apertures (Fig.1 upper row). We see here interesting patterns. For example, the FT of a square is a sinc function along the x and y-axis. Many other shapes can be applied with FT and may observe other patterns as well.

6.B. Anamorphic property of the Fourier transform

Figure 2. (below) FT of (above)sinusoid images of different frequencies (left) 1Hz (middle) 4 Hz and (right) 10 Hz)

The two points in the FT of the images are the frequencies (f and -f) of the sinusoid used (0 Hz is located between the two points). As the frequency of the sinusoid increases, the distance between the two points also increases. The points get farther away from the center as the frequency is increased.
Usual images don't have negative pixel values -- in other words the image is biased. The problem now arises, since the image is biased, is how to determine correctly the frequencies contained in the image. This is one of the powerful uses of Fourier transformation.

Figure 3. FT of 10 Hz-sinusoid images with (upper left)constant bias and (upper right)sinusoidal bias with 1 Hz frequency

The figure above shows the a biased sinusoid image with their FTs. Fig.3(upper left) is constant-biased and the Fig.3(upper right) is sinusoid-biased. As seen, the FT of the constant-biased image has three points -- the peripheral points are the frequencies (f and -f)of the image and the point at the center (at 0 Hz) accounts for the bias. The frequency of the image can now be correctly determined ignoring the bias.
What if the image has a sinusoidal bias? These biases include noise, etc. Usually, in signals and images, non-constant biases are of low frequencies. If the frequencies contained in the image is sufficiently high, we can separate easily separate the low from the high frequencies using the FT. An example is demonstrated in Fig. 3 (right). Fig.3(upper right) is a 10 Hz-sinusoid with a 1 Hz-sinusoid bias. using FT, 2 distinct frequencies were observed: (1) the one closer to the center and (2) farther away the center. The frequency closer to the center (meaning of low frequency) accounts for the bias and the farther accounts for the frequency of the image. The low-frequency can be disregard and we are able to find the frequencies contained in the image.
The obtained FT can also be multiplied with a function that to remove low frequencies. We can take its inverse FT to retrieve a better image. Note, however, that this technique cannot always serve its purpose right. Once the frequency contained in the image is close to the frequency of the bias, doing such might destroy the image.

In this part, we rotate the image the image and look into how the FTs change with the rotation.

Figure 4. (below) FTs of the (above) rotated images at : (left) 30^o (mid) 40^o and (right) 50^o

We can observe in Fig. 2 and 3, that the axis of the dots of the FT is perpendicular to the bands of the image. This observation is the same even when the image is rotated. The FTs rotate as the image is rotated. This is another property of the Fourier transformation. The axis of the FTs are just rotated and same frequencies were obtained.

Images may contain different frequencies (assuming the image has no biases). It can be a combination of sinusoids of with these frequencies (i.e summation or multiplication of sinusoids). Below are images of sinusoidal combinations and their FTs.

Figure 5. (upper row) Images formed from sinusoidal combinations and (lower row) their FTs. (upper left) z = sin(2*pi*f*X)+ sin(2*pi*f*Y) ; (upper right) z = sin(2*pi*f*X)sin(2*pi*f*Y)

The FT of the sum of 2 sinusoids (Fig. 5(lower left)) is just the sum of the FT of each sinusoid:

F_t =F{sin(2*pi*f*X)} + F{sin(2*pi*f*Y)}

This is another property of FT -- linearity. The FT of sum is the sum of FT.

This property holds regardless of the number of additions to the orignal pattern. This is illustrated in the figure below.
For the product combination,

z = sin(aX)sin(bY),

the expression can be expanded into sum of cosines given below:

z = [cos(aX - bY) - cos(aX + bY)]/2 or

z = [sin(aX - bY + pi/2) - sin(aX + bY + pi/2)]/2

This is just sum of rotated sines at opposite directions (due difference in signs of bY). This results to a pattern in Fig.5(lower right).

Figure 6. Product combination of 2 sinusoids as in Fig. 5(upper right) with additional two rotated sinusoids with frequencies 10 Hz and 25 Hz. The 10 Hz signal is rotated at 30^o. The 25 Hz-sinusoid is rotated from 30^o, 60^o and 90^o.

Using the pattern in Fig.5 (upper right) plus additional rotated patterns, since FT is a linear transform, an expected pattern is:

F_overall = F{pattern1} + F{pattern2} + F{pattern3}

Indeed, the prediction is right. The FT of the product combination is retained, the FT of the rotated 10 Hz sinusoid (as in Fig.4(lower left)), and the FT of the 25 Hz sinusoid at different rotation angles.

In this activity, I give myself a grade of 10.

I thank Jica Monsanto, Miguel Sison, Orly Tarun, and Jayson Villangca for useful discussions. Special thanks to Sison for making me realize that some of my images were wrong.