A Machine-Learning Approach to Keypoint Detection and Landmarking on 3D Meshes

Research output: Contribution to journalArticlepeer-review


We address the problem of automatically detecting a sparse set of 3D mesh vertices, likely to be good candidates for determining correspondences, even on soft organic objects. We focus on 3D face scans, on which single local shape descriptor responses are known to be weak, sparse or noisy. Our machine-learning approach consists of computing feature vectors containing $\dsn$ different local surface descriptors. These vectors are normalized with respect to the learnt distribution of those descriptors for some given target shape (landmark) of interest. Then, an optimal function of this vector is extracted that best separates this particular target shape from its surrounding region within the set of training data. We investigate two alternatives for this optimal function: a linear method, namely Linear Discriminant Analysis (LDA), and a non-linear method, namely AdaBoost. We evaluate our approach by landmarking 3D face scans in the FRGC v2 and Bosphorus 3D face datasets. Our system achieves state-of-the-art performance while being highly generic.
Original languageEnglish
Pages (from-to)146-179
Number of pages34
JournalInternational Journal of Computer Vision
Issue number1-3
Publication statusPublished - Mar 2012


  • keypoint detection, landmarking, 3D face recognition, Machine Learning, LDA, AdaBoost

Cite this