CS180: Face Morphing

Part 1

This process was developed as part of a project to morph one face into another, utilizing Delaunay triangulation to ensure smooth transitions. Last week, I created a point selection tool using matplotlib, and that served as the basis for this enhancement.

In the current implementation, I opted for a more interactive approach, using OpenCV to allow side-by-side manual selection of corresponding points from two images. I used 46 points for better accuracy, automatically adding the four corners of each image to improve the triangulation quality.

Original Images

Images with Delaunay Triangulation

The side-by-side layout of the original and triangulated images provides a clear understanding of the feature mapping between the two faces.

The Delaunay triangulation is a key part of the morphing process. It ensures that each face is divided into corresponding triangular regions, which can then be warped consistently during the morphing.

Part 2: Midway Face

In the second part of the project, we computed the "midway face" between the two images. This involved three main steps:

1. Computing the average shape: The average shape was computed by averaging each keypoint location between the two faces.
2. Warping both faces into the average shape: Using affine transformations, each triangle in the triangulation was warped into the corresponding triangle in the average shape.
3. Averaging the colors: After warping, the colors of both warped faces were averaged to create the final midway face.

Part 3: The Morph Sequence

I created a morph sequence that gradually transitions from Jackson to Joey. This sequence uses two key components: shape warping and cross-dissolving. The parameters warp_frac and dissolve_frac control the shape warping and color blending for each frame.

To create the morph sequence, the following steps were performed:

1. Warping each image into an intermediate shape configuration defined by warp_frac.
2. Blending the two warped images using dissolve_frac, resulting in a smooth transition between the two faces.

The intermediate frames were generated by varying the warp and dissolve fractions from 0 to 1 over 30 frames, creating an animated transition that smoothly transforms from one face to the other. Below is a preview of the resulting morph sequence:

This GIF demonstrates the gradual change from Jackson to Joey, showing the effective use of both the warping and color blending techniques. The Delaunay triangulation was crucial in preserving the local facial features throughout the transition, ensuring a natural morph effect.

Part 4: The "Mean Face" of a Population

I computed the "mean face" of a population using a dataset of annotated faces. I used the images in the fei.edu.br site to calculate the average face shape and appearance of the group.

The following steps were performed:

1. Compute the average face shape of the whole population.
2. Morph each of the faces in the dataset into the average shape to achieve consistent geometry.
3. Compute the average face by averaging the color values across all warped faces.

Population Mean Face

The mean face represents the average of all the faces in the dataset. Below are additional visualizations:

The first image shows my face warped to the average geometry of the population, while the second image shows the mean face of the population warped into my geometry. These visualizations help illustrate how individual facial features align with the average structure of the group.

Part 5: Caricatures: Extrapolating from the mean

I computed the "mean face" of a population of smiling faces. Then I warped my face to match it.

Population Mean Face Smiling

Smiling Jackson:

Bells and Whistles:

For my bells and whistles, I changed the smile and ethnicity of my friends face:

Here is what Kevin normally looks like:

Here is what Kevin looks like when happy: