3D Face Modeling, Analysis and Recognition
Preface ix
List of Contributors xiii
1 3D Face Modeling 1
Boulbaba Ben Amor, Mohsen Ardabilian and Liming Chen
1.1 Challenges and Taxonomy of Techniques 2
1.2 Background 3
1.2.1 Depth from Triangulation 4
1.2.2 Shape from Shading 5
1.2.3 Depth from Time of Flight (ToF) 6
1.3 Static 3D Face Modeling 7
1.3.1 Laser-stripe Scanning 7
1.3.2 Time-coded Structured Light 8
1.3.3 Multiview Static Reconstruction 11
1.4 Dynamic 3D Face Reconstruction 14
1.4.1 Multiview Dynamic Reconstruction 14
1.4.2 Photometric Stereo 17
1.4.3 Structured Light 18
1.4.4 Spacetime Faces 24
1.4.5 Template-based Post-processing 27
1.5 Summary and Conclusions 31
Exercises 33
References 35
2 3D Face Surface Analysis and Recognition Based on Facial
Surface Features 39
Faisal Radhi M. Al-Osaimi and Mohammed Bennamoun
2.1 Geometry of 3D Facial Surface 39
2.1.1 Primary 3D Surface Representations 40
2.1.2 Rigid 3D Transformations 47
2.1.3 Decimation of 3D Surfaces 49
2.1.4 Geometric and Topological Aspects of the Human Face 51
vi Contents
2.2 Curvatures Extraction from 3D Face Surface 53
2.2.1 Theoretical Concepts on 3D Curvatures 53
2.2.2 Practical Curvature Extraction Methods 56
2.3 3D Face Segmentation 57
2.3.1 Curvature-based 3D Face Segmentation 57
2.3.2 Bilateral Profile-based 3D Face Segmentation 58
2.4 3D Face Surface Feature Extraction and Matching 59
2.4.1 Holistic 3D Facial Features 60
2.4.2 Regional 3D Facial Features 67
2.4.3 Point 3D Facial Features 68
2.5 Deformation Modeling of 3D Face Surface 71
Exercises 73
References 74
3 3D Face Surface Analysis and Recognition Based on Facial Curves 77
Hassen Drira, Stefano Berretti, Boulbaba Ben Amor, Mohamed Daoudi,
Anuj Srivastava, Alberto del Bimbo and Pietro Pala
3.1 Introduction 77
3.2 Facial Surface Modeling 78
3.3 Parametric Representation of Curves 80
3.4 Facial Shape Representation Using Radial Curves 81
3.5 Shape Space of Open Curves 81
3.5.1 Shape Representation 82
3.5.2 Geometry of Preshape Space 84
3.5.3 Reparametrization Estimation by Using Dynamic Programming 86
3.5.4 Extension to Facial Surfaces Shape Analysis 88
3.6 The Dense Scalar Field (DSF) 90
3.7 Statistical Shape Analysis 94
3.7.1 Statistics on Manifolds: Karcher Mean 94
3.7.2 Learning Statistical Models in Shape Space 96
3.8 Applications of Statistical Shape Analysis 98
3.8.1 3D Face Restoration 98
3.8.2 Hierarchical Organization of Facial Shapes 101
3.9 The Iso-geodesic Stripes 103
3.9.1 Extraction of Facial Stripes 107
3.9.2 Computing Relationships between Facial Stripes 109
3.9.3 Face Representation and Matching Using Iso-geodesic Stripes 113
Exercises 114
Glossary 116
References 117
4 3D Morphable Models for Face Surface Analysis and Recognition 119
Frank B. ter Haar and Remco Veltkamp
4.1 Introduction 120
4.2 Data Sets 121
Contents vii
4.3 Face Model Fitting 122
4.3.1 Distance Measure 122
4.3.2 Iterative Face Fitting 123
4.3.3 Coarse Fitting 124
4.3.4 Fine Fitting 124
4.3.5 Multiple Components 125
4.3.6 Results 126
4.4 Dynamic Model Expansion 129
4.4.1 Bootstrapping Algorithm 131
4.4.2 Results 136
4.5 Face Matching 141
4.5.1 Comparison 141
4.5.2 Results 142
4.6 Concluding Remarks 144
Exercises 145
References 146
5 Applications 149
Stefano Berretti, Boulbaba Ben Amor, Hassen Drira, Mohamed Daoudi,
Anuj Srivastava, Alberto del Bimbo and Pietro Pala
5.1 Introduction 149
5.2 3D Face Databases 150
5.3 3D Face Recognition 157
5.3.1 Challenges of 3D Face Recognition 158
5.3.2 3D Face Recognition: State of the Art 159
5.3.3 Partial Face Matching 162
5.3.4 Comparison of State-of-the-Art Methods 168
5.4 Facial Expression Analysis 170
5.4.1 3D Facial Expression Recognition: State of the Art 171
5.4.2 Semi-automatic 3D Facial Expression Recognition 173
5.4.3 Fully Automatic 3D Facial Expression Recognition 180
5.5 4D Facial Expression Recognition 184
5.5.1 The BU-4DFE Database 186
5.5.2 3D Shape Motion Analysis 187
5.5.3 Discussion and Comparative Evaluation 192
Exercises 192
Glossary 193
References 198
Index 203