TP3: Morphage de visages (Partie A)

Moritz Bruggisser


The implementation follows the guideline declared in the lecture. The algorithm needs following datasets/variables:

1. Im1: first image which is the starting point of the morphage
2. Im2: second image to which the first image is morphed
3. img1_pts: correspondence points in image 1
4. img2_pts: correspondence points in image 2

The pipeline then was implemented as follows:

1. calculate triangulation

For each intermediate morphage stage:
2. calculate intermediate triangulation
3. for each triangle, calculate the transform in direction of Im1 and Im2
4. for each pixel within the triangles in Im1 and Im2, interpolate the color from the original image
5. add background

The triangulation was calculated using Delaunay-Triangulation. Furthermore, the triangulation was calculated on an intermediate morphing stage as it should be a good triangulation for both original images. The intermediate triangulation was calculated using the corresondence points and bases on a linear interpolation between the to images of the form:

img3_pts = warp_frac * img1_pts + (1-warp_frac) * img2_pts (I)

In this case, warp_frac was set to 0.5. The same formula later was used in step 2 for the calculation of the intermediate morphing stages. Triangulations assigns the number of the triangle to which a pixel belongs to every pixel in the image.
The idea then is to perform local transformations, i.e. transform the triangles rather than to apply a global transformation on the entire image. In this exercise, an affine transformation should be applied (step 3). This transform has 6 degrees of freedom, thus, 6 equations need to be established to resolve the system. 3 correspondences, corresponding to the 3 points of each triangle, are needed to establish the system of linear equations in order to define the transformation:

a = T * b (II)

where a are the resulting coordinates from the transform (thus img3_pts of the intermediate triangle), b are the points of the original image (img1_pts and img2_pts, respectively), and T is the affine transformation.

In Matlab, this system can be resolved by:

T = a\b (III)

For each pixel, the color value then has to be found in the original image. This is done based on a backward-transformation, i.e. for each pixel in the morphed image, the position in the original image is calculated, where the respective pixel in the new images derives from. As this backward-transformation might result in inter-pixel values (i.e. we might get a position between two pixels), the intensity-values of the morphed image are calculated based on a bilinear interpolation. In order to reduce calculation time, step 4 was applied seperately on every subset of pixels belonging to a triangle. The respective pixels could be found using mytsearch. The final pixal value is a combination between the interpolated pixel values from Im1 and Im2, where the wheigt of the two input images depends on the stage of morphing.
Finally, the background was added. I first set here 4 additional corresponding point into the corners of the image. For this points, I again calculated a triangulation and following the same approach as for the morphed objects. However, the pixel in the morphed images only was filled with the calculated backgroun intensity, if the pixel did not belong to a triangle.


First, the triangulation was set up. This looks as follows:

Where the visage mask of the first image Im1 is:

and the morphed visage mask at an intermediate stage is:

The resulting visage morphing looks as follows:

Further examples

The same algorithm was tested with different sets of images. First, a devil is transformed into a bull:

source bull: source devil:

Another example is the transformation between animals, in this example the morphing of a lion into a tiger:

source lion:,fl_progressive,h_1200,q_90,w_2000/v1/178274/The-best-top-desktop-lion-wallpapers-hd-lion-wallpaper-11_xjbx08.jpg source tiger:
A third example:

source trump: source frog:

Own examples

The approach of morphing is not limited to visages, but can also be adapted to objects. First, two different shoes are used for morphing:

Another example is morphing of a kettle into a cup:


As seen, the morphing algorithm works for visages as well as for objects. The contours are transformed into each other as demanded.
However, two requirements have to be fulfilled in order to achieve this results. First, an adequate number of correspondences has to be defined in both image. Definition of the adequate number is not always easy to define and highly depends on the object. For example, morphing of devil/bull was achieved based on 49 correspondences instead of 43 as for the visages.
Second, care has to be taken that the arrangement of corresponding points in both images is comparable and of equal order. Equal order is mandatory as only this proceeding ensures a correct morphing.
However, in the examples above, some deficiencies can be recognized. A first deficiency relates to the background/background definition. Background definition is not always very obvious, as for example for the lion and the tiger. As the mane of the tiger fills a large part of the image, I considered some amount of it as background. However, this approach was not too wise. The same can be seen for the devil, who's hair I considered as background, like I did the body of the bull. This reduces the morphing effect.
Another problem affects the blending of the intermediate morphing stages. I chose the weights according to the proportion of the stage, defined by warp_factor in equation (I). However, the second images appears to fast while the first image disappears to slowly in morphing. A wiser decision might be found here.

Crédits supplémentaires

Morphing of hybrid image


In this example, a hybrid image as intermediate stage in morphing is created. That for, a high-pass filtered image (Einstein) is morphed into a low-pass filtered image (Monroe). While Einstein is visible at first, his portrait is morphed into the one of Marylin Monroe, where the intermediate step corresponds to the hybrid image.
Filtering of the images was done as described in TP2 (same thresholds for the filters) and image morphing was done using the algorithm from above.


The input images as well as the results from morphing are presented in the figures.

original Einstein original Marilyn
high-pass filtered Einstein, normalized low-pass filtered Marilyn, normalized
animation intermediate stage


The morphed hybrid images is not as good as the hybrid image from TP2. The region in the image, within which the hybridization works fine, is limited to the area defined by the correspondence points, while for the background, the hybrid image is affected by the parameter defining aggregation of the two images, namely the variable dissolve_frac. The morhped and hybridized image region, which is reduced to the visage, thus, visibly stands in the foreground at the intermediate stage, showing distinct edges to the background. This lowers the effect of hybridization.

Polar coordinates


The approach here is the same as for cartesian coordinates. Homogenous polar coordinates are of form [theta, rho, w].

Vector-field based Morphing


Instead of transforming triangles, where the idea is to perform local transformation, one could find specific transformations for every pixel. The workflow looks as follows:

1. find displacements between the corresponding points, i.e. find desired displacement between the corresponding point in the original images and the target image (morphed image)
2. calculate distances from every pixel to every correspondence point 3. calculate a vector field by interpolation
4. specify the transformation for every pixel

Step 1 results in the displacement vector between the corresponding points in the images. For each correspondence pair, the respective vector can be calculated as:

[dx, dy] = [(x_p1,target-x_p1,original), (y_p1,target - y_p1,original)];

As vector field interpolation should be weighted, the distance for every pixel to every correspondence point was calculated in step 2. This distances are used in step 3 for the interpolation, where the weights for every displacement vector of the correspondence points are chosen as the inverse of the squared difference to a pixel, normalized by the total sum of distance. Finally, the calculated displacement was applied for every pixel.


The figure shows the result from pixel-wise displacement based on interpolated vector field. Here, I used the portraits from task 1.


The implemented algorithm works in principle, the portraits are morphed. However, interpolation of the vector field shows some deficiencies, resulting in some hard breaks where portraits show some breaks. Clearly visible are the correspondence points around which morphing works fine. On the other hand, only parts within the visages are affected, while the general outline of the heads transforms well.

TP3: Morphage de visages (Partie B)


In a second part, we want to calculate an average face based on the Utrecht data set which contains a total set of 131 portrait photographs of 68 men and 63 women. Furthermore, the average male and female face is calculated in order to masculinize and feminize my own portrait.
The same averaging approach was also applied on the images of the class mates in order to retrieve the average class face.

Average face


The pipeline comprises of the following basic steps:

1. identify face characteristics/features
2. calculate the average face
3. morph all images into the average shape
4. calculate mean of all morphed images

For a large data set as the Utrecht data set, a manual feature selection is not feasible. Thus, the dlib-library was deployed which automatically detects features (step 1) which are later used for the correspondences and the triangulation.
For the set of correspondences, the mean position of each correspondence point was calculated by averaging all positions (step 2).
For face morphing, the core algorithm was the same as for Part A of this TP. Only difference regards the shape to which the face is morphed, which is the average shape of all faces rather than an interpolated shape at intermediate stage between two faces. As morphing is done separately for all faces in the input data set, the results are averaged and stacked as final step.


Class data

First, the manually selected correspondence points have been used for the face calculation. This lead to following results:

average face shape from own correspondence points average class shape after morphing

dlib-detector on our images lead to following results:

average face shape from dlib average class shape after morphing

Utrecht data

average face shape as detected by dlib average shape from morphing


Although the faces are not perfectly aligned within the images and might contain asymmetries, the average face shapes are of symmetric shape. Plots of the average shapes ( triangulations) also reveals dlib to detect correspondences within the actual face only, while our points also include the shape of the head. This results in a slightly different average face for the class portraits compared to the result when using our points. When using our correspondences, the hairs are morphed as well and the intensity values are explicitly sampled in the original images, while for the dlib-correspondences, this section of the head is not morphed and the values there are derived from the background. As the faces are not perfectly aligned, hair and background are mixed in the hair section.
This is also true for the Utrecht data set. All parts of the head outside the detected face are blurred. Furthermore, a slight shortcoming of my implementation becomes visible here as the transition from face to background is very distinct.

Bells and Whistles

As the Utrecht data set contains information on the expression of the person, i.e. whether the person smiles or not, the average smiling and non-smiling face can be calculated. The approach is exactly the same as depicted above, only difference regards the portraits based on which the average face is calculated.


A total number of 58 smiling faces was available, for which following average face shape and face could be retrieved:

average face shape as detected by dlib average shape from morphing


A total number of 73 non-smiling faces was evaluated, for which following average face shape and face could be retrieved:

average face shape as detected by dlib average shape from morphing
The differences between the smiling and the non-smiling faces are clearly visible both in the average face shape (triangles) as well as in the average face.



Utrecht data furthermore provides information on the gender of the faces. This information can be used to calculate the average male and female face, respectively.
Furthermore, average male and female shape can be used to masculinize and feminize my portrait. To do so, triangulation of my face was morphed into the mean shape of all men and women, respectively.
However, the problem was, that the Utrecht images have different sizes than my portrait and scale of the faces are not equal to the one of my portrait. Thus, I first brought Utrech images and my portrait to a width of 900px. In order to retrieve a comparable scale, I slightly enlarged triangulation of the Utrecht data set. A third problem was a misalignment between Utrecht triangulation and triangulation on my portrait. I thus shifted my portrait in order to align it to the Utrecht data.


The mean male faces look as follows.

average male face shape as detected by dlib average shape from morphing
The average female faces are the following:

average female face shape as detected by dlib average shape from morphing

my female face my male face


The differences in facial characteristics between men and women are striking: The average female face characteristics are much softer then the ones of men.
The problem is, that my face is not correctly located anymore. I ascribe this to a badly performed rescaling/ shifting step, which is described above and which was necessary as the images are of different size. Furthermore, scales of faces in the Utrecht data set show variances. Thus, despite I tried to align the images, I did anything wrong. For this reason, I was neither able to overlay my face to the average Utrecht man/woman.