Procrustates rotation

  1. Introduction

Artificial intelligence technique has become extremely popular within the decades. With emerging applications of this technique, our life has changed dramatically with all different smart tools. For instance, since 2013 Apple has introduced the Touch ID feature, which allows users to unlock Apple devices using their fingerprint. This smart feature can record and store the images of users’ fingerprints into a “fingerprint pool”, determine whether a new entered fingerprint belongs to the pool, and unlock the device if fingerprint matches.

When using the Touch ID feature, it is impossible for us to place a finger on the home button with same angle every time. Therefore, a question which arises, is how to match fingerprints with different angles?  The solution, which sounds simple, is to rotate the fingerprints till they are comparable; however, in practice, it contains multiple steps with complicated algorithms to ensure a good result. The objective of my project is to illustrate the shape rotation technique using a simple example.

  1. Procrustes Rotation Project

The project is to rotate the given 20 shapes (2-D) and align them using the Procrustes Superimposition method. The 20 shapes, which look like bones, are plotted in different colors as below.

Figure 1. 20 Shapes before Procrustes Rotation

Each shape is represented with 100 ordered points. Example of Shape 1 is illustrated in Figure 2.

Figure 2. Plot of Shape 1

The coordinates of each points are stored in the raw data. For example, data of first 10 points in Shape 1 is as below.

Figure 3. Coordinates of First 10 Points in Shape 1

The Procrustes Superimposition (PS) is a shape analysis method. It aims to obtain a similar placement and size of shapes by optimally translating, rotating and uniformly scaling the objects, with minimum shape differences [1]. According to Figure 1, all 20 shapes have similar sizes and are centered at the same point, but with different rotation angles. Therefore, we only need to conduct the Procrustes rotation step in the project.

To simplify the problem, I first try to rotate one shape to the other, which is considered as a reference position. Figure 4 shows the original positions of the two shapes before rotation; specifically, the shape with blue dots is used as reference. With Procrustes rotation, the red shape is rotating toward the blue one.

Figure 4. Two Shapes before Rotation

The result is shown in Figure 5 and the shape with red empty dots is after rotation.

Figure 5. Two Shapes after Rotation

The next step is to rotate the 20 shapes together toward a single reference position, where the reference position can be randomly chosen from the given 20 shapes. Examples of rotation results with different reference positions are shown below. Based on the rotation results, we can conclude that the Procrustes rotation is a valid tool for shape rotation with good results.

 

 

Figure 6. Examples of Rotation Results (Reference Shape = 3, 5, 12, 19)

  1. Applications

Given the Apple Touch ID example, the fingerprint identification system needs to rotate our fingerprints before making comparisons. In this way, the Procrustes rotation may be applied in this situation. However, the Procrustes rotation is only part of the identification process, which aims to match a fingerprint with other saved fingerprints. Generally, such identification process is called image registration or image matching.

The main task of image registration is to align two or more images acquired at different time, sensors, or viewpoints. It has widely applications in medical imaging analysis, weather forecasting and computer visions [2]. For instance, doctors will take a brain fMRI image prior to an operation, to help plan the procedure, and then take another one after the procedure. By aligning these two images, the doctor can determine whether the operation is successful. Another interesting application is the creation of panoramic pictures using the iPhone. We can take multiple pictures to cover the whole viewing space by moving the phone around, and the system will automatically stitch the images together to produce a panoramic picture.

There are various image registration algorithms that applies in different problems. One is called feature-based method, which focuses on comparing the features of different images, such as points (landmarks), curves, or surfaces, etc. [1, 3]. This method is typically applied when the image structure provides more information than image intensity. Further, the feature-based methods can handle the images with distortions, since they rely on the image features, not every voxel in the image. Based on the image features, the feature-based method can also be classified into landmark-based method, curve-based method, hybrid method, etc.

When we try to describe a shape by locating a finite number of points on the objects, these points are called landmarks [1]. The data we used in the project is a set of landmarks of bone-like shapes. The locations of landmarks can be determined by object’s intrinsic properties, such as anatomical landmarks of human brain, or by mathematical calculations. The landmark method aims to align images based on the properties of selected landmarks. Back to the fingerprint identification example, the system firstly finds the ridge ending points and bifurcations points from the input fingerprint images [4]. Then it uses these points as landmarks for further matching analysis.

Figure 7. Ridge Ending and Bifurcation

When drawing lines to connect the landmarks, we get multiple curves that can also describe the shape of an object in the image. Using curves as features in image registration is called curve-based method. Since the curves result from connecting finite points, this curved-based method pays more attention on the sequences or relationships between points, rather than single points. This method is typically applied when the curves provides more information than the points, such as handwriting recognition [5]. Part of the recognition process is to align the curves of a letter-like object with the curves of a letter.

When combining the landmark and curve features together, the method is called hybrid method. This method can handle more complex image matching problems since it analyzes information from both points and curves. For instance, face identification is a complicated process, where face has both point-based features, location of eyes, nose, and mouth, and curve-based features, face shape, nose shape, hair style, etc. Therefore, the hybrid method will give a more accuracy result.

  1. Conclusion and Future Studies

In the shape rotation project, we successfully rotated and aligned 20 shapes by implementing the Procrustes rotation method. The data used in the project are the coordinates of landmarks, which are clean and well organized. However, in the real situation, the raw data, such as a set of real pictures, can be messy with lots of notices and contain tremendous information. Therefore, the challenge is how to extract effective features, landmarks or curves, from these pictures while maximizing contain useful image information. To be specific, in the face identification system, some interesting topics may include:

  1. How to detect the location of eyes, nose, and mouth?
  2. How to distinguish face from the background?
  3. How to align face images taken from different viewpoint? How to align the left side to the right side?

 

 

 


Reference

  1. Dryden, I. L., & Mardia, K. V. (1998). Statistical shape analysis (Vol. 4). Chichester: Wiley.
  2. Wyawahare, M. V., Patil, P. M., & Abhyankar, H. K. (2009). Image registration techniques: an overview. International Journal of Signal Processing, Image Processing and Pattern Recognition, 2(3), 11-28.
  3. Hong, G., & Zhang, Y. (2007, July). Combination of feature-based and area-based image registration technique for high resolution remote sensing image. In Geoscience and Remote Sensing Symposium, 2007. IGARSS 2007. IEEE International (pp. 377-380). IEEE.
  4. Kommini, C., Asadi, S., & Ellanti, K. (2011). Scale and Rotation Independent Fingerprint Recognition. Chaitanya Kommini et al,/(IJCSIT) International Journal of Computer Science and Information Technologies, 2(4), 1764-1773.
  5. Frenkel, M., & Basri, R. (2003, July). Curve matching using the fast marching method. In International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition (pp. 35-51). Springer, Berlin, Heidelberg.
# function to rotate shape x to y
rotate <- function(x,y){
  u <- svd(x%*%t(y))$u
  v <- svd(x%*%t(y))$v
  
  d <- det(t(v),u)
  o_hat <- v%*%diag(c(1,d))%*%t(u)
  result <- o_hat%*%x  
  return (result)
}

###################
# load data
###################
all_shapes <- as.matrix(read.table("shape.txt"))
shape <- function (i){
  return (all_shapes[(i*2-1):(i*2),])  
}

# display all shapes in the same plot
for(i in 1:20){
  #i=4
  par(new=T)
  plot(shape(i)[1,],shape(i)[2,],xlim = c(-0.15,0.15), ylim = c(-0.15,0.15), col=i)
}


# show one plot and one point as example
i=4
plot(shape(i)[1,],shape(i)[2,],xlim = c(-0.15,0.15), ylim = c(-0.15,0.15), col=i)
rbind(shape(i)[1,],shape(i)[2,])
pt=3
points(shape(i)[1,pt],shape(i)[2,pt],col='red',pch=16)


# show two shapes of rotation as example
a=3
plot(shape(a)[1,],shape(a)[2,],xlim = c(-0.15,0.15), ylim = c(-0.15,0.15), col='red',pch=16)
b=5
points(shape(b)[1,],shape(b)[2,],xlim = c(-0.15,0.15), ylim = c(-0.15,0.15), col='blue',pch=16)
r=rotate(shape(a),shape(b))
points(r[1,],r[2,],xlim = c(-0.15,0.15), ylim = c(-0.15,0.15), col='red')





# allign all shapes to a randomly chosen shape
a <- ceiling(20*runif(1))

plot(shape(a)[1,],shape(a)[2,],xlim = c(-0.15,0.15), ylim = c(-0.15,0.15), col=a)

for(i in 1:20){
  par(new=T)
  r <- rotate(shape(i),shape(a))
  plot(r[1,],r[2,],xlim = c(-0.15,0.15), ylim = c(-0.15,0.15), col=i)
}

Leave a Reply