Abstract

The main aim of this assignment is image alignment. For each Image, we have access to its three channels, Red, Blue, and Green. By combination of these channels, a colored image is generated. Since the three chandells are not aligned, the generated image is not clear. In order to produce a clear image, we should firstly devise an algorithm to align these channels to each other. The resultant image should be cropped automatically to remove artifacts from the borders.



Image Alignment

To produce a colorized and clear image from its red, green, blue channels, one should combine these channels. Since these channels are not aligned, the colorized image is vague. To align these channels, firstly align the red channel to the blue channel, and then align the green channel to the blue channel. Since these channels only shifted (no rotation, nor scaled), we just need to find the best pair of x, y for the red channel, and another pair for the green channel. To specify the two channels (e.g. red, and blue) are correctly aligned or not, we can use an intensity based metric like SSD(Sum of Squared Difference), NCC(normalized cross correlation).

A source channel (e.g. Red channel) should be transformed by x, and y in order to align with a target channel (e.g. Blue channel), then compute SSD or NCC to asses the quality of this alignment. In fact, The SSD considered as an objective function which its constraints are two small intervals (e.g. [-20:20]) for x,and y. Our purpose is finding a pair of x, and y that makes this objective function minimum. For finding this pair of x, and y, we implement an algorithm (called makealign) that exhaustively search for different values of x, and y, and return the best values for x, and y. It is worth to mention that if one channel needs a huge linear transformation (beyond the specified intervals) for alignment, this algorithm will fail. Therefore, I think that one possible way to resolve this problem is extracting a set of distinctive features(by using SIFT) form the channels, and then find x, and y where these features matched. In this assignment, I used NNC as objective function, which should be maximized.

This algorithm pretty aligns low-resolution images, while it does not produce an aligned image for a high-resolution image. Therefore, we have to utilize the image pyramid. In other words, as we down to the pyramid, the resolution of the image becomes higher. So, We devise an algorithm that finds best x_i, and y_i for image in resolution level i (called I_i) by calling makealign . For the image in the next level of resolution (I_i+1), we shifted this image by 2*x_i, and 2*y_i, then call makealign. This algorithm repeated for the images in all resolution levels and finally the algorithm is finished by finding x, and y for the image in the highest resolution level.

Automatically Cropping Process

In order to crop an aligned image, we firstly find all lines in the borders of the image (left, right, up, and bottom). We used 10% of image's height for up and bottem borders, and 10% of image's weigth for the left and right borders. In other words, we know some artifacts appear in the image's borders, which these artifacts are straight line. Hence, Finding these lines was done by Hough transformation, which is able to find straight lines in an image. We sorted these lines by their length in descending order. In this stage, we thought that the longest line is the best choice for cropping, but this is not correct, as we experienced. For example, consider the left border of this image, the longest line is not the best choise for cropping.

An aligned Image
The Left corner
Therefore, for each line in the borders, we investigated two tiny parts. The left part (called P1) is 5 pixels before the line, and the right part (called P2) is 5 pixels after the line. Then the intensity of P1, and P2 are compared. For the left border, and the up border, If mean(P1) is smaller than mean(P2), we select this line for cropping. If the artifacts in the borders are not like a straight line, this algorithm is not able to remove them, as you can see in the result section.

Experimental Results

To asses the aforementioned algorithms, we tested them on various images. In all cases, these algorithm correctly aligned images.

Low Resolution Images

OriginalAlignedCropped
Bes Bes Bes
Red offset [14,4], Green offset [6,2]
BesBesBes
Red offset [9,-1], Green offset [4,1]
Bes Bes Bes
Red offset [5,5], Green offset [2,3]
Bes Bes Bes
Red offset [13,1], Green offset [6,1]
Red offset [4,3], Green offset [1,2]
Red offset [2,0], Green offset [6,0]
Red offset [13,-1], Green offset [1,-1]
Red offset [11,1], Green offset [5,1]
Red offset [11,1], Green offset [5,1]

Additional Low Resolution Images

Red offset [12,3], Green offset [6,3]
Red offset [5,2], Green offset [1,-1]
Red offset [15,3], Green offset [7,2]
Red offset [11,0], Green offset [4,1]
Red offset [-2,10], Green offset [5,0]

High Resolution Images

Red offest [90,37], Green Offset [38,18]
Red offest [90,37], Green Offset [38,18]
Red offest [80,32], Green Offset [35,8]
Red offest [46,15], Green Offset [12,7]
Red offest [131,37], Green Offset [60,29]
Red offest [42,3], Green Offset [16,1]
Red offest [4,14], Green Offset [-21,5]
Red offest [110,60], Green Offset [49,47]

Additional High Resolution Images

Red offest [97,20], Green Offset [41,20]
Red offest [100,0], Green Offset [43,3]
Red offest [119,63], Green Offset [48,36]
Red offest [129,47], Green Offset [58,32]

Mahdieh's Image

This image was taken from screen of my office's computer. I could not find any interesting object in my lab, so I have to use this image!
Red offest [40,-10], Green Offset [14,54]