Activity 11: Camera Processing

In this activity, we observed the effect of different white balancing options in cameras. Also, we applied the white patch and gray world algorithms on improperly white balanced images. Both RGB objects and objects with same color but different hues were used. It was found that for all images used, it is better to use the white patch algorithm.

For the first part, the same set of images were captured under different white balancing options in the camera. The white balancing options used were cloudy, daylight, fluorescent, fluorescentH, and tungsten. Capturing the images under these conditions is like illuminating the object with a light source with the same color temperature as the white balancing conditions. The images are shown below. The image captured under automatic white balancing is also included for comparison. Note that in the images the order is as follows

automatic white balanced
cloudy daylight fluorescent
fluorescentH tungsten

RGB

GREEN

From the images it can be observed that image taken under the tungsten condition is obviously improperly white balanced. The supposedly white paper appeared blue.

The white patch and gray world algorithms is then applied to these images to for white balancing. Each image has RGB values. In the white patch algorithm, we get a patch from the image that is supposedly white. The RGB value of this patch is used to divide the RGB value of the whole image such that Rn = R/Rw, Gn=G/Gw and Bn=B/Bw where Rn,Gn, Bn are the white balanced RGB values, R,G,B are the original RGB values and Rw,Gw,Bw are the RGB values of the patch.

For the gray world algorithm, it is assumed that the average color is gray. The key term here is "average". To operate the gray world algoirthm, we used average of the R,G, and B values and used this to divide the RGB values of the original image.

The following images are the result of the white patch and gray world algorithms on the images.

RGB IMAGES
Cloudy
Daylight
Fluorescent
FluorescentH
Tungsten

Notice that better images result from the white patch algorithm. The gray world algorithm produces a very bright image. This is maybe because white dominates in the image where it was applied.

GREEN IMAGES

Cloudy
Daylight
Fluorescent
FluorescentH
Tungsten

Like in the RGB images, the white patch algorithm obviously works better. Again, brighter images are observed in the gray world algorithm. Notice though that yellow green notebook became yellow when both algorithms were applied. This may be because the yellow color of the notebook is stronger than that of its green color.

The code:
stacksize(4e7)
rgbwp=imread("F:\186-11\wp.jpg");
rgb=imread("F:\186-11\green\tungsten.jpg");

rw=rgbwp(:,:,1);
gw=rgbwp(:,:,2);
bw=rgbwp(:,:,3);

r=rgb(:,:,1);
g=rgb(:,:,2);
b=rgb(:,:,3);

//white world
Rw = sum(rw)/(size(rgbwp,1)*size(rgbwp,2));
Gw = sum(gw)/(size(rgbwp,1)*size(rgbwp,2));
Bw = sum(bw)/(size(rgbwp,1)*size(rgbwp,2));


//gray world
Rg = mean(r);
Gg = mean(g);
Bg = mean(b);


rr=r/Rw;
rg=g/Gw;
rb=b/Bw;

gr=r/Rg;
gg=g/Gg;
gb=b/Bg;

chuva = [];
chuva(:,:,1)=rr;
chuva(:,:,2)=rg;
chuva(:,:,3)=rb;
chuva(chuva>1) = 1;
//scf(0), imshow(chuva);
//scf(1), imshow(rgb);

chuva2 = [];
chuva2(:,:,1)=gr;
chuva2(:,:,2)=gg;
chuva2(:,:,3)=gb;
chuva2(chuva2>1) = 1; //pin values to 1
//scf(1), imshow(chuva2);

imwrite(chuva,"F:\186-11\result\green\tungstenw.jpg");
imwrite(chuva2,"F:\186-11\result\green\tungsteng.jpg");

Notice that the image values are all pinned to 1.

For this activity, I give myself a 10/10 since I believe I was able to perform the task asked for this activity. Thank you to Neil and Earl for the discussion we had.

Activity 10: Preprocessing Text

For this activity, we used imaging techniques we've learned so far to process a handwritten document. The goal for the first part is to isolate each letter from each other and from the lines of the form. To remove the lines, we designed a filter such that it blocks the lines. Since the lines are repetitive in the form, they will have intense lines in the Fourier space. Moreover, since I cropped the image such that only horizontal lines are seen, we used a vertical filter. Prior to filtering, the image was rotated using Gimp to make the horizontal lines straight. The original image (cropped and rotated), the filter used, and the resulting image are shown below.
Notice the lines in the form are not visible as it was before although some lines can still be seen in the resulting image. The next part is to isolate the letters from each other by using morphological operators. Prior to that, the image is inverted since the operations work on the white part of the image. By examining the histogram, a threshold value was obtained and was applied to the image to further isolate the text with the background. Afterwhich, morphological operations were applied to isolate each letter and make it 1 pixel thick each. For this particular case I used

erode(erode(dilate(erode(inviconv,se),se),se),se2)

since this gave me the best result so far. The binarized image and the resulting image (after morphological operations were implemented) are shown below.
Notice in the figure that not all letters are 1 pixel thick and some letters are not visible anymore. This is the downside of the morphological operations on the letters. Since in the binarized image the thickness of each letter varies significantly, when morphological operations are applied, some letters (or parts of it) which are originally thin, are erased. A better image is expected if we have almost the same thickness for the letters :P

To make the handwritten more readable, we have to compensate the thickness. Using this operator instead

ero=erode(dilate(erode(inviconv,se),se),se)

give a more a readable result and the erased the remaining lines visible in the binarized object. Although some of you may object that it is not readable enough, my claim is that thatpart is subjective :P

For the second part, we are to perform the correlation of the word "DESCRIPTION" in the text. Assuming that the word has the same size in the image althroughout, strictly speaking, only 3 intense bright spots (corresponding to high correlation) should be observed in the Fourier Transform (FT) since "DESCRIPTION" was appeared only 3 times. However if we are to look at the FT, we have other bright spots. This is because there are white parts that can acoomodate the whole area of "DESCRIPTION". This results to higher correlation and hence a bright spot. An example is the logo on top of the form.
To verify this result, I cropped the image such thatthe logo is annot included anymore. Here we observed the 3 intense spots as what was expected.

For this activity, I give myself a 7/10 since I was not able to isolate the letters from each other, and make a readable 1 pixel thick letter. For me, this seems to be the hardest activity.

Many thanks to Ma'am Jing, Gilbert and Neil for the insights :)

Activity 9: Binary Operations

In this activity, we are to estimate the size of one punched paper by examining an ensemble. By applying the closing and opening operators in Scilab, we tried to separate nearly touching circles and clean the edges of circles. From here, we plotted the obtained areas and the peak which corresponds to more frequent areas within the ensemble is taken as the area of an individual circle.

The figure below is the image of the ensemble of punched papers. This image is further subdivided into 256x256 subimages. For our discussion, we will refer only to one subimage. All subimages undergone the same morphological operations.

ORIGINAL

SUBIMAGE

The subimage is binarized. The proper threshold value is obtained from its histogram. In depth discussion of which is found in the previous activities. If we are to consider a subimage with only one circle, thresholding is enough. However, an ensemble of circles needs further operations since overlapping of circles may occur. Another reason for further operation is that the area of each circle may vary. The opening and closing operators were implemented on the subimages. Bwlabel in scilab renders a particular color to a blob. A blob is anything white in the binarized image. So long as one circle is connected even with only 1 pixel, bwlabel reads that as one blob. Note that the structuring element used is a circle with a diameter of 4 pixels.

After the subimage undergone the operation it looked like the one below. The subimage (original and binarized) was also posted for comparison.
In the binarized image we can see that the circles were well separated from the background. However, there are still circles that are overlapping. After applying the operator we could see that we were able to separate one circle as one blob which corresponds to a particular area. Note that the extracted areas were the areas of blobs. By tallying the areas we obtained the most occuring area for each subimage. This was done for all 13 subimages. The resulting histogram (below) shows that the most occuring area is at around 541 pixels. To verify this area, I isolated one circle and measured its diameter. The diameter obtained was 26 which orresponds to an area of 530. This is near the value extracted from the histogram.

HISTOGRAM

For this activity I give myself an 8/10 since I am not sure if the operation I used is enough.
Collaborators:
neil, gary, gilbert

Activity 7: Enhancement in the Filter Domain

In this activity, the convolution theorem is used to enhance images. In the first part of the activity, we explored the convolution theorem. The succeeding parts deals with ridge enhancement, line removal, and canvas weave modeling and removal from images. All of which were done by exploiting the convolution theorem in the Fourier space.

A. Convolution Theorem
For this part, the Fourier transform (FT) of images were done. The images used are two dots of 1 pixel each, circles of different radii, squares of different area symmetric about the center. A Gaussian function of varying variance also was used instead of dots. The following are the images described above and their corresponding FTs.
It can be observed from the above images that with increasing area for circles and squares, the FT becomes smaller. It can also be seen that the FTs are sine and a sinc for circles and a squares, respectively. For FTs for the Gaussian functions decreases in area with increasing variance (or effectively increasing area).

B. Ridge Enhancement
Using the FT of an image, we can create a filter such that only the high frequencies can pass. This is what we did here. High frequencies in your image correspond to bright areas in your Fourier space. These high frequencies are repetitive in your image. Particularly for a fingerprint, these are the ridges. This is the reason why the particular filter below was used. The following shows the original fingerprint, the enhanced fingerprint, the filter used and the FT of the original image. Notice that in the reconstructed image, the blotches were minimized. they were acutally spread out, thus a more enhanced ridges were obtained. However, there is a downside with this enhancement. The reconstructed image was blurry on other areas. This maybe due to the filtered frequencies that should not be filtered out.

C. Line Removal
For this part, The inverse of part B was done. Here, we actually filtered out the bright areas in the image. This bright areas correspond to the lines you like to remove in the image. We designed a filter such that these high frequencies cannot pass. Indeed, we obtained an image without the lines. Again, the following are the image,its FT, the filter used, and the reconstructed image.
D. Canvas Weave Modeling and Removal
The same technique as that in C is applied here. The repetitive patterns of the canvas weave correspond to high frequencies in the Fourier space. To remove the canvas weave in the image, we designed a filter such that these high frequencies are blocked. In the canvas weave modeling, the filter (only the black dots) used is basically the canvas weave patterns. So taking its inverse FT is like modeling the canvas weave itself. However, results showed that the canvas weave model is angled either 90 or 180 degrees with respect to the original canvas. Results are shown below.

Canvas Weave Removal


Canvas weave Model

*** To enhance the image properly, make sure that the FT of the image coincides with the filter design. The problem I encountered was I cannot get an enhanced image. This is because my filter did not actually filtered the frequencies I wanted since it did not coincide with the FT of the image.

I give myself an 8/10 for this activity since I am not sure if I did the ridge enhancement properly. Remember that a blurry image was formed.

Thank you to Gary and Gilbert for suggesting to use "imwrite" instead of" imshow" in obtaining the FTs and designing the filters.

Activity 6: Properties of the 2D Fourier Transform

In this activity, the properties of the 2D Fourier Transform (FT) were investigated. For the first part, the FT of a square, donut, square annulus, two slits, and two dots were obtained. The following images show the images with its corresponding FT's.
The second part is to create a 2d sinusoids with different frequencies and obtain their FT's. Results showed that as you increase the frequency, the spacing between the dots in the FT's also increases. This is because at higher frequencies, the spacing between the sinusoids decreases. This is equivalent to an increase in the spacing of the delta function (or in this case, the dots) in the frequency domain. Note that the FT of a sinusoid is a delta function.

On the next part of the activity, we investigated the effect of adding a constant bias on the sinusoid. From the results, it can be seen that adding a negative or positive value of bias does not change the FT as long as the magnitudes are equal. Note that frequency = 4 for all was used.
When the sinusoids are rotated from 0-120, their FTs also rotates, which is in accrodance to the Introduction part of the manual. [1] (frequency = 4)
The product of two sinusoids is a checkerboard with circles or ellipses. Circles are obtained when both the coefficients for X and Y are the same, otherwise, an ellipse. The FTs produced are dots arranged in a square (for circles) and in rectangles (for ellipses) . Notice also that the distribution of the dots has the same shape as the component of the checkerboard. What follows are the images of the generated sinusoids and their corresponding FT's.
For the last part of the activity, the FTs of combinations of the sinusoids with varying rotating angle and the product of two sinusoids were first predicted. Superposition of the waves were basically done. From the linearity property of FT, the FT of the sum of two waves is just the FT of the first wave plus the FT of the second wave. Here, we expect to get the same result as the previous only this time they are rotated. Results verified this linearity property.
For this activity, I give myself an 8/10. Thank you to Raffy for introducing the 'mesh()' function. It was fun to look at the sinusoids in 3D! :D Thank you also to Ma'am Jing for the code :)

**Scilab code can be found in Reference 1.
Note that to see better results, open the image on a new tab :)

Reference:
Dr. Maricor Soriano. AP 186 Activity 6 Manual

Activity 5: Fourier Transform Model of Image Formation

For this activity, Fourier Transform was implemented on images. The first part of the activity is to get the FFT of a circle and the letter 'A'. Shown below are the original image, the shifted FFT of that image, and the FFT of its FFT. Notice that the third image is just a reconstruction of the original image, only this time it is inverted. This is a property of the inverse FFT - that the inverse FFT is just the same as the forward FFT with the image inverted. [1] This is not readily seen in the circle since it is symmetric in all directions.


The second part of the activity deals with convolution of two images. The system resembles an imaging system with the circle serving as an aperture and the text 'VIP' as the object. The radius of the circle is varied and the following results were obtained.

r=5

r=10
r=15
r = 48.5

As can be seen, the larger the radius of the aperture, the better the reconstruction. This is because the number of frequencies that can pass through is dependent on the size of the aperture. For larger apertures, more frequencies can pass through. Hence a better reconstruction is obtained as opposed to that of apertures with smaller radii.

The third part of the activity is to find the correlation of the letter 'A' in the sentence "THE RAIN IN SPAIN STAYS MAINLY ON THE PLANE." To do this, the FFT of 'A' is multiplied element per element with the FFT of the text. The result shows that the letter A's in the text are highlighted because of high correlation.
The figure above shows the text (1st), the letter 'A' with the same font style and font size, and the resulting image when their FFTs are multiplied element per element.

The last part of the activity shows edge detection using the convolution integral. Four patterns were used (horizontal, vertical, diagonal, and spot). The following results were obtained.
The pattern allows only certain components to be seen. For example, the reconstruction for the horizontal pattern allows only almost horizontal components. Most vertical components are missing in the horizontal pattern, most horizontal components are missing in the vertical pattern, other vertical and horizontal patterns are missing in the diagonal pattern, and most components are present in the spot pattern.



The code (taken from Reference 1):

//first part
I=imread("C:\Documents and Settings\2004-28298\Desktop\activity 5\circle.bmp");
Igray=im2gray(I);
FIgray= fft2(Igray);
fc=abs(fftshift(FIgray))/max(abs(fftshift(FIgray)));
ffc=abs(fft2(FIgray))/max(abs(fft2(FIgray)));
imwrite(fc,"C:\Documents and Settings\mimie\Desktop\activity 5\fc.bmp");
imwrite(ffc,"C:\Documents and Settings\mimie\Desktop\activity 5\ffc.bmp");


//2nd part
o=imread("C:\Documents and Settings\mimie\Desktop\activity 5\vip.bmp");
I=imread("C:\Documents and Settings\mimie\Desktop\activity 5\30r.bmp");
lgray=im2gray(I);
ogray=im2gray(o);
fl=fftshift(lgray);
fo=fft2(ogray);
col=fo.*(fl);
icol=fft2(col);
fimage=abs(icol);
fimagenorm=fimage/max(fimage);
imwrite(fimagenorm,""C:\Documents and Settings\mimie\Desktop\activity 5\vip30r.bmp");


//3rd part
t=imread("C:\Documents and Settings\mimie\Desktop\activity 5\text.bmp");
a2=imread("C:\Documents and Settings\mimie\Desktop\activity 5\A2.bmp");
tgray=im2gray(t);
a2gray=im2gray(a2);
ft=fft2(tgray);
fa2=fft2(a2gray);
cft=conj(ft);
cta2=fa2.*(cft);
fcta2=fft(cta2);
ta2=fftshift(abs(fcta2))/max(fftshift(abs(fcta2)));
imwrite(ta2,"C:\Documents and Settings\mimie\Desktop\activity 5\ta2.bmp");


//last part
i=imread("C:\Documents and Settings\mimie\Desktop\activity 5\vip.bmp");
igray=im2gray(i);
pattern1=[-1,-1,-1; 2,2,2; -1,-1,-1];
pattern2=[-1,2,-1; -1,2,-1;-1,2,-1];
pattern3=[-1,-1,-1; -1,8,-1;-1,2,-1];
pattern4=[2,-1,-1;-1,2,-1;-1,-1,2];

c1=imcorrcoef(igray,pattern1);
c2=imcorrcoef(igray,pattern2);
c3=imcorrcoef(igray,pattern3);
c4=imcorrcoef(igray,pattern4);

c1norm=c1/max(c1);
c2norm=c2/max(c2);
c3norm=c3/max(c3);
c4norm=c4/max(c4);

imwrite(c1norm,"C:\Documents and Settings\mimie\Desktop\activity 5\c1norm.bmp");
imwrite(c2norm,"C:\Documents and Settings\mimie\Desktop\activity 5\c2norm.bmp");
imwrite(c3norm,"C:\Documents and Settings\mimie\Desktop\activity 5\c3norm.bmp");
imwrite(c4norm,"C:\Documents and Settings\mimie\Desktop\activity 5\c4norm.bmp");

**Remember to comment the part of the code you don't need.

Many thanks to Ma'am Jing for providing the code which guided me through this activity. Thank you also to Kaye and Miguel for the discussions we had, and to Raffy for helping me. I give myself a 9/10 for this activity because I did what was expected from the activity.

[1] Dr. Maricor Soriano. Activity 5 Manual.

Activity 4: Enhancement by Histogram Manipulation

For this activity, a poor contrast image is enhanced by histogram manipulation by following the steps below. The code (if applicable) used and the results accompany the steps.

1. Get an image from the internet or from own collection. Crop if necessary.

I=imread('C:\Documents and Settings\mimie\Desktop\New Folder (2)\activity 4 images\crop.bmp');

2. Change image to grayscale and get its histogram and corresponding Cumulative Distribution Function (CDF).
I=im2gray(I);
gcrop=I/max(I);
imwrite(gcrop, "C:\Documents and Settings\mimie\Desktop\New Folder (2)\activity 4 images\gcrop.bmp");

xsize=size(I,1);
ysize=size(I,2);

x = matrix(0:255,[1,256]);
y = x.*0;
a = 1;

for i = 0:255
m = find(I== i);
y(a) = length(m);
a = a + 1;
end

cs=cumsum(y);
csnorm=cs/max(cs);


3. Backproject the CDF of original image to a linear CDF. Get resulting image, its histogram and its cdf.

//linear desired cdf

for i=1:xsize
for j=1:ysize
n=find(x==I(i,j));
newI(i,j)=cs(n);
end
end

4. Backproject to a nonlinear CDF and get resulting image, histogram, and CDF of reconstructed image.
//nonlinear

z = [0:255];
g = tanh(30.*(x-128)./255);
g = (g - min(g))./2;

plot(x,g);
for i = 1:xsize
for j = 1:ysize
n1 = find(x == I(i,j)); //get index of x from image
y1 = csnorm(n1); //get corresponding y value
n2 = find(g<=y1); //equate with y in desired cdf
newI2(i,j) = csnorm(n2(max(n2)));
end
end

The original image has poor contrast since its pixel values does not cover the entire 0-255 values. By backprojecting its original CDF to a linear CDF with pixel values from 0-255, a better contrast is obtained. When we say backproject, it means that it follows the new CDF, but not perfectly, of course. This can be seen from the CDF of the reconstructed image using a desired linear CDF. When the nonlinear CDF (for this case, hyperbolic tangent) is used, the quality of the image degraded. Notice that the histogram of the reconstructed image is narrower than that of original image. Hence a poorer image contrast was obtained. We can conclude in this activity that image contrast of an image can either be improved or degraded depending on the CDF to which these images are backprojected.

For this activity, I give myself an 7/10. Thank you to Ma'am Jing, Ma'am Gay, Raffy, Kaye, and Neil for the help.

***

• Note that to get the histogram and CDF of the reconstructed image, we have to multiply the image to 255 since we had it normalize to 255 before displaying it.
  
• to change the color of plot to green : plot(x,y,,['g']) may apply to other colors as well :)

References
1. Dr. Maricor Soriano. Activity 4 manual.
2. image taken from: http://yrekahigh.com/Portals/1/YrekahighPics/Misc_%20Pictures/Old%20Yreka%20High.bmp