Pose changing from picture to picture is one of the bottlenecks toward full automatic face recognition, which results in the performance dramatically decreasing.

• Frontalization

Pose-invariant face recognition is achieved by pose normalization. First, normalize a non-frontal face image to a virtual frontal view (hereinafter Frontalization), which is then fed into the more developed frontal face recognizers and finally get the matching results.

• Solution #1: Statistical Affine Transformation (SAT)

Assuming the face can be viewed as a 2D-plane at a distance, the face rotation can be modeled by the affine transformation. To model the pose-invariant well, the face region is divided into three disjoint rectangles, the sketch map is illustrated as Fig.1. The map across the specific and frontal poses can be inferred for each small rectangle region. In test stage, when given a face image, first determine the coarse pose class and then the three groups of affine transformation parameters computed previously of the corresponding pose can be used to the three regions respectively and synthesize the virtual frontal view. See our PCM2003 paper or contact Ms.Xiujuan Chai.

• Solution #2: Pose Normalization based on 3D Sparse Deformable Model (3D SDM)

A direct solution to pose-invariant face recognition is the pose normalization. Our aim is to reconstruct a specific 3D face from single face image and then get the frontal view by graphic rendering. We proposes a novel method to efficiently reconstruct 3D face from single face image based on a statistical deformable model regressed through 2D geometry formed by some facial landmarks. Using a 3D face database, PCA is performed to construct a statistical deformable model of the dense 3D shape, from which a sparse version is derived to model the statistical deformation of the 3D points corresponding to some predefined facial landmarks in the image. Given a face image with coarse pose estimated, the landmarks are first localized by LTC-AAM method to form the 2D geometry constraints, based on which an iterative regressive procedure is then exploited to estimate the model and pose parameters of the sparse 3D deformable model. The parameters are then reverted to construct the dense 3D shape, followed by texture mapping to reconstruct the 3D face. Experimental results convincingly show the effectiveness for the pose-invariant face recognition. See our AVBPA2005 paper or contact Ms.Xiujuan Chai

• Solution #3: Pose Normalization based on Local Linear Regression (LLR)

The variation of facial appearance due to the viewpoint (/pose) degrades face recognition systems considerably, which is well known as one of the bottlenecks in face recognition. Among the possible solutions is the strategy generating virtual frontal view from any given non-frontal view. By formulating this kind of solutions as a prediction problem, we propose a simple but efficient novel Local Linear Regression (LLR) method, which can generate the virtual frontal view of a given non-frontal face image. The motivation of our LLR lies in the basic observations that the corresponding local facial regions of the frontal and non-frontal view pair satisfy linear assumption much better than the whole face region. This can be explained easily by the fact that a 3D face shape is composed of many local planar surfaces, which satisfy naturally linear model under camera projection. In LLR, we simply partition the whole non-frontal face image into multiple local patches and apply linear regression to each patch for the prediction of its virtual frontal patch. To appear in FG2006 or contact Ms.Xiujuan Chai.