Activity 12: Color Image Segmentation

For this activity, we are to select a region in the image with a particular color. First we have to transform the color space into a normalized chromaticity coordinates. To do this the following transformations are used:
Since 1=r+g+b, b=1-r-g. Therefore we can transform the color space (3-D) into two coordinate space (2-D), with the brightness associated with I. The image and patch used is shown below:

(Image taken from http://www.magicbob2000.com/resources/Cups%20&%20Balls%20Blue.jpg)

Patch

Two methods were used in this activity - parametric and nonparametric. For parametric a Gaussian Probability Distribution is used. The equation of which is given by
for the red values. The same is true for the green values (p(g)). The probability that a given pixel is inside the region of interest is dictated by prob=p(r)*p(g). For the nonparametric method, histogram backprojection is used. The pixel value in the histogram is used to backproject the value for a particular pixel. The histogram , result of parametric and nonparametric estimation of the image used is shown below:

Histogram

Parametric

Nonparametric

Although both estimations would suffice, a better segmentation is achieved with parametric estimation. It is also interesting that even the red "thingy" is well seen for parametric estimation. For the image used, the objects were separated from the background and from each other. Note however that we are only interested in segmenting the blue cups. Strictly speaking we did not segment the red "thingy". It just so happened that the background is violet (both with red and blue) which are the colors of the object that is why a gray color can be observed. Recall that when segmenting, what happens is that the region of interest becomes white. Any other color has different value.

Code:
stacksize(4e7);
chdir("C:\Documents and Settings\mimie\Desktop\186-12"); patch = imread("patch.jpg"); image = imread("cups.jpg"); ave = patch(:,:,1)+patch(:,:,2)+patch(:,:,3)+1e-7;
R = patch(:,:,1)./ave;
G = patch(:,:,2)./ave;
B = patch(:,:,3)./ave;

r = R*255;
g = G*255;

ave2 = image(:,:,1)+image(:,:,2)+image(:,:,3)+1e-7;
r2 = image(:,:,1)./ave2;
g2 = image(:,:,2)./ave2;
b2 = image(:,:,3)./ave2;
f = zeros(256,256);
for i=1:size(r,1)
for j=1:size(r,2)
x = abs(round(r(i,j)))+1;
y = abs(round(g(i,j)))+1;
f(x,y) = f(x,y)+1;
end
end
//imshow((frequency+0.0000000001));
//mesh(frequency);
//xset("colormap",jetcolormap(256));

\\\\parametric
rmean = mean(R);
rdev = stdev(R);
gmean = mean(G);
gdev = stdev(G);
rprob = (1/(rdev*sqrt(2*%pi)))*exp(-((r2-rmean).^2)/2*rdev);
gprob = (1/(gdev*sqrt(2*%pi)))*exp(-((g2-gmean).^2)/2*gdev);
prob = rprob.*gprob;
prob = prob/max(prob);
scf(0); imshow(image);
scf(1); imshow(prob,[]);

\\\\nonparametric
R2 = r2*255;
G2 = g2*255;
s = zeros(size(image,1),size(image,2));
for i = 1:size(R2,1)
for j = 1:size(R2,2)
x = abs(round(R2(i,j)))+1;
y = round(G2(i,j))+1;
s(i,j) = f(x,y);
end
end
scf(1); imshow(log(s+0.000000000001),[]);
scf(2); imshow(image);

Note:
Comment the parametric to use nonparametric


For this activity, I give myself an 8/10 since I'm not sure if I fully understand the activity.

Activity 11: Camera Processing

In this activity, we observed the effect of different white balancing options in cameras. Also, we applied the white patch and gray world algorithms on improperly white balanced images. Both RGB objects and objects with same color but different hues were used. It was found that for all images used, it is better to use the white patch algorithm.

For the first part, the same set of images were captured under different white balancing options in the camera. The white balancing options used were cloudy, daylight, fluorescent, fluorescentH, and tungsten. Capturing the images under these conditions is like illuminating the object with a light source with the same color temperature as the white balancing conditions. The images are shown below. The image captured under automatic white balancing is also included for comparison. Note that in the images the order is as follows

automatic white balanced
cloudy daylight fluorescent
fluorescentH tungsten

RGB

GREEN

From the images it can be observed that image taken under the tungsten condition is obviously improperly white balanced. The supposedly white paper appeared blue.

The white patch and gray world algorithms is then applied to these images to for white balancing. Each image has RGB values. In the white patch algorithm, we get a patch from the image that is supposedly white. The RGB value of this patch is used to divide the RGB value of the whole image such that Rn = R/Rw, Gn=G/Gw and Bn=B/Bw where Rn,Gn, Bn are the white balanced RGB values, R,G,B are the original RGB values and Rw,Gw,Bw are the RGB values of the patch.

For the gray world algorithm, it is assumed that the average color is gray. The key term here is "average". To operate the gray world algoirthm, we used average of the R,G, and B values and used this to divide the RGB values of the original image.

The following images are the result of the white patch and gray world algorithms on the images.

RGB IMAGES
Cloudy
Daylight
Fluorescent
FluorescentH
Tungsten

Notice that better images result from the white patch algorithm. The gray world algorithm produces a very bright image. This is maybe because white dominates in the image where it was applied.

GREEN IMAGES

Cloudy
Daylight
Fluorescent
FluorescentH
Tungsten

Like in the RGB images, the white patch algorithm obviously works better. Again, brighter images are observed in the gray world algorithm. Notice though that yellow green notebook became yellow when both algorithms were applied. This may be because the yellow color of the notebook is stronger than that of its green color.

The code:
stacksize(4e7)
rgbwp=imread("F:\186-11\wp.jpg");
rgb=imread("F:\186-11\green\tungsten.jpg");

rw=rgbwp(:,:,1);
gw=rgbwp(:,:,2);
bw=rgbwp(:,:,3);

r=rgb(:,:,1);
g=rgb(:,:,2);
b=rgb(:,:,3);

//white world
Rw = sum(rw)/(size(rgbwp,1)*size(rgbwp,2));
Gw = sum(gw)/(size(rgbwp,1)*size(rgbwp,2));
Bw = sum(bw)/(size(rgbwp,1)*size(rgbwp,2));


//gray world
Rg = mean(r);
Gg = mean(g);
Bg = mean(b);


rr=r/Rw;
rg=g/Gw;
rb=b/Bw;

gr=r/Rg;
gg=g/Gg;
gb=b/Bg;

chuva = [];
chuva(:,:,1)=rr;
chuva(:,:,2)=rg;
chuva(:,:,3)=rb;
chuva(chuva>1) = 1;
//scf(0), imshow(chuva);
//scf(1), imshow(rgb);

chuva2 = [];
chuva2(:,:,1)=gr;
chuva2(:,:,2)=gg;
chuva2(:,:,3)=gb;
chuva2(chuva2>1) = 1; //pin values to 1
//scf(1), imshow(chuva2);

imwrite(chuva,"F:\186-11\result\green\tungstenw.jpg");
imwrite(chuva2,"F:\186-11\result\green\tungsteng.jpg");

Notice that the image values are all pinned to 1.

For this activity, I give myself a 10/10 since I believe I was able to perform the task asked for this activity. Thank you to Neil and Earl for the discussion we had.

Activity 10: Preprocessing Text

For this activity, we used imaging techniques we've learned so far to process a handwritten document. The goal for the first part is to isolate each letter from each other and from the lines of the form. To remove the lines, we designed a filter such that it blocks the lines. Since the lines are repetitive in the form, they will have intense lines in the Fourier space. Moreover, since I cropped the image such that only horizontal lines are seen, we used a vertical filter. Prior to filtering, the image was rotated using Gimp to make the horizontal lines straight. The original image (cropped and rotated), the filter used, and the resulting image are shown below.
Notice the lines in the form are not visible as it was before although some lines can still be seen in the resulting image. The next part is to isolate the letters from each other by using morphological operators. Prior to that, the image is inverted since the operations work on the white part of the image. By examining the histogram, a threshold value was obtained and was applied to the image to further isolate the text with the background. Afterwhich, morphological operations were applied to isolate each letter and make it 1 pixel thick each. For this particular case I used

erode(erode(dilate(erode(inviconv,se),se),se),se2)

since this gave me the best result so far. The binarized image and the resulting image (after morphological operations were implemented) are shown below.
Notice in the figure that not all letters are 1 pixel thick and some letters are not visible anymore. This is the downside of the morphological operations on the letters. Since in the binarized image the thickness of each letter varies significantly, when morphological operations are applied, some letters (or parts of it) which are originally thin, are erased. A better image is expected if we have almost the same thickness for the letters :P

To make the handwritten more readable, we have to compensate the thickness. Using this operator instead

ero=erode(dilate(erode(inviconv,se),se),se)

give a more a readable result and the erased the remaining lines visible in the binarized object. Although some of you may object that it is not readable enough, my claim is that thatpart is subjective :P

For the second part, we are to perform the correlation of the word "DESCRIPTION" in the text. Assuming that the word has the same size in the image althroughout, strictly speaking, only 3 intense bright spots (corresponding to high correlation) should be observed in the Fourier Transform (FT) since "DESCRIPTION" was appeared only 3 times. However if we are to look at the FT, we have other bright spots. This is because there are white parts that can acoomodate the whole area of "DESCRIPTION". This results to higher correlation and hence a bright spot. An example is the logo on top of the form.
To verify this result, I cropped the image such thatthe logo is annot included anymore. Here we observed the 3 intense spots as what was expected.

For this activity, I give myself a 7/10 since I was not able to isolate the letters from each other, and make a readable 1 pixel thick letter. For me, this seems to be the hardest activity.

Many thanks to Ma'am Jing, Gilbert and Neil for the insights :)

Activity 9: Binary Operations

In this activity, we are to estimate the size of one punched paper by examining an ensemble. By applying the closing and opening operators in Scilab, we tried to separate nearly touching circles and clean the edges of circles. From here, we plotted the obtained areas and the peak which corresponds to more frequent areas within the ensemble is taken as the area of an individual circle.

The figure below is the image of the ensemble of punched papers. This image is further subdivided into 256x256 subimages. For our discussion, we will refer only to one subimage. All subimages undergone the same morphological operations.

ORIGINAL

SUBIMAGE

The subimage is binarized. The proper threshold value is obtained from its histogram. In depth discussion of which is found in the previous activities. If we are to consider a subimage with only one circle, thresholding is enough. However, an ensemble of circles needs further operations since overlapping of circles may occur. Another reason for further operation is that the area of each circle may vary. The opening and closing operators were implemented on the subimages. Bwlabel in scilab renders a particular color to a blob. A blob is anything white in the binarized image. So long as one circle is connected even with only 1 pixel, bwlabel reads that as one blob. Note that the structuring element used is a circle with a diameter of 4 pixels.

After the subimage undergone the operation it looked like the one below. The subimage (original and binarized) was also posted for comparison.
In the binarized image we can see that the circles were well separated from the background. However, there are still circles that are overlapping. After applying the operator we could see that we were able to separate one circle as one blob which corresponds to a particular area. Note that the extracted areas were the areas of blobs. By tallying the areas we obtained the most occuring area for each subimage. This was done for all 13 subimages. The resulting histogram (below) shows that the most occuring area is at around 541 pixels. To verify this area, I isolated one circle and measured its diameter. The diameter obtained was 26 which orresponds to an area of 530. This is near the value extracted from the histogram.

HISTOGRAM

For this activity I give myself an 8/10 since I am not sure if the operation I used is enough.
Collaborators:
neil, gary, gilbert