In this homework, we aim to implement the algorithm that can morph one face image into another face image by the way of triangulation, affine transformation and color dissolving. Speicifically, we begin with defining several corresponding feature points in the source image and dest image respectively. Next, we compute the weighted average positions of the feature points and triangulize them. The resulting triangulation is used to locate the original pixels mapped into the pixels of intermediate image by the affine transformation. Once we find these correspondence between the source, dest and intermediate image, the pixel colors of the intermediate image will be obtained by computing the weighted average of source and dest corlors. The above process proceeds until the desired number of intermediate images with different weights is reached.
Morphing:
(1) For the morphing implemention (see the code 'morph.m'), we first calculate the weighted average of feature points of source image (matrix 'S') and feature points of dest image (matrix 'D') to get the intermediate shape (matrix 'I') using the warping weight parameter 'warp_frac'.
(2) In the second step (see the code 'triangulation.m'), a delaunay triangulation (matrix 'Tri') is computed according to intermediate shape defined by matrix 'I'.
(3) In the third step (see the code 'affine_transformation.m'), we iterate through the intermediate triangulation set (matrix 'Tri'), for each triangle, the transformation between the intermediate and source (and dest) shape is calculated, which results in a transformation matrix 'T_is' (and 'T_id'). In detail (see the code 'transformation.m'), every transformation involves in solving a system of six linear equations corresponding to the six parameters defined in the transformation matrix 'T_is' (and 'T_id').
(4) In the final step (see the code 'dissolve.m'), once we have all the transformation matrices 'T_is' (and 'T_id'), all the source (dest) pixels mapped from the pixels in the intermediate shape by 'T_is' can be located, and the weighted mean (measured by 'dissolve_frac') color computed from the souce pixel and dest pixel mapping from the same intermediate pixel are used to represent the color of intermediate pixel. Furthermore, if the location of souce (dest) pixel is not a integer, the interplation (by the 'interp2' function, or by the 'nearest' point) of the colors of neigbouring pixels is used to approximate the color of that pixel. Thus, one dissolved image is obtained and we continue until the expected number of intermediate images is achived.
Average Face:
As far as computing average face concerned, the idea is almost the same except that instead of computing the intermediate image between two face images, we compute the intermediate shape using all the face images. Then, we calculate the mean color by mapping the corresponding pixel from each of source images to the same pixel in the intermediate image and computing the average mapped color. Specifically, the differences are:
(1) In the first step (see the code 'average_face.m'), the average intermediate shape (matrix 'I') is computed using all the source shapes (3-dimensional array 'P') with each taking the weight of 1/9.
(3) In the third step (see the code 'average_face_affine_transformation.m'), we use a 4-dimensional array 'T' to store all the transformation matrices from intermediate triangles to source triangles.
(4) In the fourth step (see the code 'average_face_dissolve.m'), we dissolve all the source images using the all the transformation information in 'T'.
When morphing the images, the two parameters 'warp_frac' and 'dissolve_frac' should both range from 0 to 1. The ordering and step size of increasing the sequence of parameters is important.
In the first case, we try to increase the 'dissolve_frac' much faster than 'warp_frac'. e.g. The 'dissolve_frac' reaches to 1 while 'warp_frac' reaches only to 0.1. We see that the features are not well aligned when it begins to dissolve, however, in this example this phenomenon seems not so clear because the two face images have the same pose and the similar shape, and it will be much more obvious in the situation where there is big difference in the poses.of the faces.
(the first case): one intermediate image and the gif image are in the center
In the second case, if the 'warp_frac' goes up much faster than 'dissolve_frac'. e.g. The 'warp_frac' already reaches to 0.9 when the image starts to dissolve. As a result, the two faces are aligned too much before dissolving, which leads to more distorted and discontinued intermediate faces.
(the second case): one intermediate image and the gif image are in the center
To alleviate the above situations, we first increase the parameter 'warp_frac' to 0.25 by 0.025 per step to generate 10 images while keeping the parameter 'dissolve_frac' (=0) unchanged. Starting from 0.25, we then increase both parameters alternately until 0.75 with respective step size of 'warp_frac' 0.025 and 'dissolve_frac' 0.0125 to generate the another 100 images. Finally, we increase the parameter 'warp_frac' to 1 by 0.025 per step to obtain the remaining 10 images. The result as follows:
(the better case): the gif image are in the center
For the mean face (left), it is just an ordinary human that is much closer to a male, since the number of the male, which is 7, is much greater than the number of female, which is 2. If we impose more weight on the 2 female students, the resulting image (right) shows more female characteristics.
(left): the average of all faces (right): the weighted average of all faces
(1) This is an example of two faces with different poses, and the parameter 'warp_frac' is increaseed a lot (to 0.5) before we increase the parameter 'dissolve_frac'.
(Beckham and Victoria):
An ilustration of morphing between two objects with different poses is given below. Note that if two objects differ a lot in the pose and orientation, then there will be significant changes in the sizes of the intermediate shpes, which is due to the affine transformation. Furthermore, we observe that this method can only deal with the convex hull of the ojbects, it can not handle the situation very well where the objects are not convex. For example, the pear is almost convex while the bulb is not convex aournd the narrow part.
(pear and bulb):
In addition to suffering the convexity problem, another scenario that this algorithm does really poorly is when the feature points between the source and dest are not all one-to-one correspondences. For example, the example below demonsrates several intermediate images during morphing the full body of a chimpanzee with a significant different orientations. It can be seen that the part of right thigh is blocked by the right arm in the source image, but it is not the case in the dest image. Likewise, the part of left thigh is blocked by the right leg in the dest image though it is not the case in the source image. We see that the face part is ok but the lower body of chimpanzee is severely distorted. This does not happen for the case of human face morphing, since there is a clear one-to-one correspdonce between two human faces if the directions are similar.
(Chimpanzee):
An illustration of morphing between two objects with different poses is given below. Note that if two objects differ a lot in the pose and orientation, then there will be significant changes in the sizes of the intermediate shpes, which is due to the affine transformation. Furthermore, we observe that this method can only deal with the convex hull of the ojbects, it can not handle the situation very well where the objects are not convex. For example, the pear is almost convex while the bulb is not convex aournd the narrow part.
(2) This is an example of morphing between a human face and a animal
(Eastwood and monkey):
These are another two examples of morphing between a human face and a animal with different poses
(Carrey and monkey):
(Dogg and dog):
The following transformation is from a human face to a cartoon chracter
(Bibber and cartoon):
(3)As we have have more images of male students than that of female students, we show a example of maculinizing using the idea of average face. In order to masculinize a female, we start by computing a average face of all the male students which represents an ordinary male human face. After that, we can obtain a masculinized female face by calculating a weighted average between a particular female face and the ordinary male face. Here we illustrate an intermideate image in the masculinization.
(masculinization): the left is a particular female face, the right is a ordinary male face, the center corresponds to the masculinized face
