Introduction

The Sergei Mikhailovich Prokudin-Gorskii Collection features color photographic surveys of the vast Russian Empire made between ca. 1905 and 1915. Frequent subjects among the 2,607 distinct images include people, religious architecture, historic sites, industry and agriculture, public works construction, scenes along water and railway transportation routes, and views of villages and cities. An active photographer and scientist, Prokudin-Gorskii (1863-1944) undertook most of his ambitious color documentary project from 1909 to 1915.
The main purpose of this assignment is to align the three channels of Red, Green and Blue. Consequently, implementation of an algorithm that does the alignment efficiently is a challenge regarding the size and resolution of the images. As a result, the single scale and multi scale algorithms are presented to align low resolution and high resolution images respectively.

Single Scale Algorithm

In the single scale algorithm, the blue channel is used as a reference for alignment. The differences of a window with the size of 30x30 are calculated to find the best match position of the other two channels regarding the blue one, which is the position with the minimum error or difference. The offsets are detemined based on the minimum error found in the differences. Then, the red and green channels are shifted by the offset values and concatenated to make up the RGB image.
Although it is common to use SSD which is the sum of squared differences, I got better results just by finding the minimum difference.

JPEG Images

Single Scale Alignment Cropped Borders
Red Offset = [1,9]
Green Offset = [1,3]		 Vertical Edges
Horizontal Edges
Red Offset = [5,5]
Green Offset = [3,2]		 Vertical Edges
Horizontal Edges
Red Offset = [1,13]
Green Offset = [1,6]		 Vertical Edges
Horizontal Edges
Red Offset = [3,4]
Green Offset = [2,2]		 Vertical Edges
Horizontal Edges
Red Offset = [1,6]
Green Offset = [1,2]		 Vertical Edges
Horizontal Edges
Red Offset = [1,12]
Green Offset = [1,1]		 Vertical Edges
Horizontal Edges
Red Offset = [2,4]
Green Offset = [1,1]		 Vertical Edges
Horizontal Edges
Red Offset = [1,11]
Green Offset = [1,5]		 Vertical Edges
Horizontal Edges
Red Offset = [4,14]
Green Offset = [2,6]		 Vertical Edges
Horizontal Edges

Multi Scale Algorithm

Since single scale algorithm is time consuming, it is not recommended to be applied to images with high resolution. To overcome this problem, the large image is rescaled to four different scales by factor of 2 as in a pyramid. The offsets are calculated with different window sizes for each step, and passed to the next level by a recursive call.
There are four images that are not aligned well, which I think the reason could be the repeating pattern that avoids to match correctly; however, there are images with repeating pattern that have been aligned well like the mountain covered in trees. The first river image has a good result in tif format but it not aligned well here as it is in jpg format.

High Resolution Images

Red Offset = [12,25]
Green Offset = [8,4]		 Red Offset = [22,27]
Green Offset = [16,20]
Red Offset = [32,37]
Green Offset = [21,26]		 Red Offset = [4,7]
Green Offset = [6,10]
Red Offset = [15,42]
Green Offset = [7,24]		 Red Offset = [19,44]
Green Offset = [16,29]
Red Offset = [5,23]
Green Offset = [4,12]		 Red Offset = [12,9]
Green Offset = [8,4]
Red Offset = [20,33]
Green Offset = [13,16]

Additional High Resolution Images

00904u.tif
Red Offset = [18,32]
Green Offset = [14,15]		 00906u.tif
Red Offset = [4,26]
Green Offset = [8,14]
00907u.tif
Red Offset = [4,31]
Green Offset = [5,16]		 00917u.tif
Red Offset = [9,32]
Green Offset = [9,16]
00912u.tif
Red Offset = [8,16]
Green Offset = [8,8]		 00913u.tif
Red Offset = [9,27]
Green Offset = [8,14]
01665u.tif
Red Offset = [32,35]
Green Offset = [20,16]		 00553u.tif
Red Offset = [12,25]
Green Offset = [8,4]
00936u.tif
Red Offset = [33,30]
Green Offset = [22,13]		 00938u.tif
Red Offset = [21,20]
Green Offset = [13,9]
00940u.tif
Red Offset = [20,26]
Green Offset = [15,13]		 00926u.tif
Red Offset = [19,12]
Green Offset = [13,10]

My Photos

I took three photos from each scene with my camera in different resolutions. Then I had the three channels by splitting the pictures. The first image was aligned with multi scale algorithm due to its higher resolution and the two other, with single scale algorithm.

>
R G B Red Offset = [1,9]
Green Offset = [1,3]
>
R G B Red Offset = [4,4]
Green Offset = [4,4]
>
R G B Red Offset = [4,8]
Green Offset = [4,7]

Bells & whistles

Cropping Unpleasant Surrounding

The first method that came to my mind to discard unpleasant borders caused by the alignment process or the leftovers around each color channel, was to crop the image by the total offset values calculated in alignment for both R and G channels. As shown in the image below, the main problem of this method is that it is kind of a blind cropping. Since channels have white or black areas around them initially, even larger than the offset values, such a cropping is not effective.

Before Cropping Cropped by Total Offset Values

With that unacceptable results, I came up with another method that uses 3x3 Sobel filter to detect edges in the image, because in a neighboring around borders we can see drastic change in pixel values. Then by setting a threshold all weak edge pixels are eliminated. From the image of horizontal edges, the row with the most number of strong edges is chosen as the border. The same procedure is done for columns with the vertical edges image. The horizontal and vertical edge image and the result cropped image is shown below.

Vertival Edges Horizontal Edges
Cropped Using Sobel Filter

Smoothing Images

In order to remove pulse noise and smooth images without losing details such as borders, I tried 3x3 median filter but the result was blurred. Furthurmore, it is a time consuming process as it sorts an array of nine elements in each step. So by eliminating corner elements of the filter, the result was improved.
3x3 Median Filter Cross Shaped Median Filter

References

Library of Congress Website