Skip to main content

Multi-Biometrics for Security & Surveillance

Researchers: Marios Savvides, Keshav Thirumalai Seshadri, Shreyas Venugopalan, Taihei Munemoto

Research Area: Security of Cyber-Physical Systems

Abstract

We plan to continue to enhance CyLab Biometrics Lab capabilities particularly towards un‐cooperative biometric identification. There are 3 large main component research thrusts of this project that will be proposed to be developed which will use leveraged resources. These 3 components are broken up to several more sub‐components which lead to about 8 research tasks.

1) 3D Face Reconstruction from single 2D Input Images Using Morphable Models for 2D Pose Correction

2) 2D Pose Correction Using Active Appearance Models (AAMs)

3) 2D Facial Landmark detection Using Shape Models to aid 3D Model reconstruction and AAM fitting process

1. 3D Face Reconstruction from single 2D Input Images Using Morphable Models for 2D Pose Correction

In many security applications, especially applications which require less co‐operative subjects, such as terrorist watch‐list applications where the suspects s are trying to evade surveillance cameras, we have to develop technologies that can maximize the use of the current surveillance architecture. Face recognition is the most widely sought biometric modality to achieve this goal, however due to the nature of the camera infrastructure and scenario, non‐ideal face captures are collected which must be pre‐processed to be fed to face recognition algorithms. This is “THE” problem in face recognition, more so than facial expression and illumination variation issues. We propose to tackle this approach using two different approaches each have its own merits. The first approach we propose to use is research & development in 3D Morphable Models, i.e. given a 2D facial image, we first detect facial feature landmarks and then build a 3D facial model by extrapolating the 3D depth information (shape of the face) which we use to rotate the face to any pose required to match against a database. The enrollment database may be frontal or maybe at different angles (e.g. if the enrollment image is not from a mug‐shot but from a newspaper article or some other source). The proposed approach will have significant impact in the fight against Terrorism, particularly for identifying possible terrorists. In order to achieve this task, we need to acquire a large database of 3D faces that are representative of 3D facial characteristics (we do NOT require a 3D scan of the person in 2D input image).

2. 2D Pose Correction Using Active Appearance Models (AAMs)

There is a great need for real‐time (30+fps) pose correction, which can be achieved using Active Appearance Models (AAMs). We propose to extend our current work by focusing on making the fitting process robust to unseen faces and illumination. Current AAM system is challenged to unseen face shapes, and illumination changes which affect the registration and the resulting pose correction, thus there are several core components of the AAM which we will be changing to make the fitting process more robust. We have explored and developed multi‐view AAMs in the past year which have shown significant performance over single global AAMs.

3) 2D Facial Landmark detection Using Shape Models to aid 3D Model reconstruction and AAM fitting process

We will explore and develop ASM models for facial and iris features for enhancing the fitting process of AAMs. This approach will also help the 3D morphable model facial feature registration process.