Skip to content

feat(antispoofing): replace IR presence check with real liveness classification#108

Open
Tunahanyrd wants to merge 9 commits into
TickLabVN:mainfrom
Tunahanyrd:feat/ir-liveness-model
Open

feat(antispoofing): replace IR presence check with real liveness classification#108
Tunahanyrd wants to merge 9 commits into
TickLabVN:mainfrom
Tunahanyrd:feat/ir-liveness-model

Conversation

@Tunahanyrd

Copy link
Copy Markdown
Contributor

Summary

The previous IR anti-spoofing path only verified that a face-shaped bounding box was visible in the IR frame. This meant a printed photo or a screen replay of a face would pass the IR check, since the YOLO model cannot distinguish a real face from a photo in IR.

This PR replaces that with a proper two-stage IR liveness pipeline and fixes a critical logic inversion in the shared FaceAntiSpoofing classifier.


Changes

auth/face/antispoofing/face_as.cc — Critical logic fix + hardening

  • Fixed logic inversion: spoof_cls == 0spoof_cls == 1. The original code treated the REAL class (index 0) as a spoof — every genuine face was rejected, every spoof attempt was accepted. This is now corrected.
  • Added output shape validation: GetTensorTypeAndShapeInfo().GetShape() is called before argmax. If a model with an unexpected number of output classes is loaded, a std::runtime_error is thrown immediately (caught by the surrounding try/catch, returning false). This prevents silent out-of-bounds reads.
  • Moved argmax into anonymous namespace to avoid ODR conflicts with other translation units.
  • Added comment clarifying that the model emits post-softmax probabilities ([REAL, SPOOF]).

auth/face/antispoofing/ir_camera_as.h / ir_camera_as.cc — Two-stage IR liveness pipeline

  • Added antispoof_model_path and antispoof_threshold parameters to checkAntispoofByIRCamera.
  • After YOLO detects a face, the cropped bounding box is passed to FaceAntiSpoofing (MobileNetV3). The image has identical R, G, B channels replicated from the native GREY V4L2 format, which is what the model was trained on.
  • Fails closed when the anti-spoofing model file is missing or the path is empty.
  • Saves failed IR frames under the ir_spoof label when debug mode is on.
  • Removed the TODO comment that tracked the missing liveness check.

auth/face/antispoofing/antispoof_check.cc — Wiring

  • Forwards antispoof_model_path and antispoof_threshold from FaceMethodConfig to the IR async task.
  • Added // SAFETY: comment documenting the raw-pointer lifetime contract for ICameraCaptureSession* captured in the lambda.

auth/face/models/mobilenetv3_antispoof.onnx — New model

  • Added the MobileNetV3 anti-spoofing model (tracked via Git LFS).
  • Input: 1×3×128×128 — RGB channels replicated from IR greyscale.
  • Output: [REAL, SPOOF] post-softmax probabilities.
  • Source: facenox/face-antispoof-onnx

app/src/app/configuration/-components/face/FaceSetting.tsx — UI fix

  • The anti-spoofing threshold slider is now shown when IR camera is configured, even if the RGB AI model is disabled. Previously the slider only appeared when anti_spoofing.enable was true, so IR-only users had no way to adjust the liveness classifier threshold from the UI.

docs/IR camera.md — Documentation

  • Updated to describe the new two-stage pipeline (detect + classify).
  • Removed outdated language about presence-only detection.

README.md

  • Updated Comparison table: reflects that IR anti-spoofing is now real liveness classification.
  • Added mobilenetv3-antispoof to the References section.

How It Works

IR frame captured
      │
      ▼
YOLOv8n-face detects bounding box
      │  no face → FAIL (deny)
      ▼
Crop face region
      │
      ▼
MobileNetV3 classifies [REAL=0, SPOOF=1]
      │  spoof score ≥ threshold → FAIL (deny)
      ▼
IR check PASSED ✓

Both the IR pipeline and the RGB AI model run in parallel (std::async). Authentication is granted only when all enabled methods pass.


Testing

Verified on Fedora 64-bit with a RealSense IR camera (/dev/video2):

Scenario Result
Real face in front of IR camera ✅ PASS
Printed A4 photo held in front of IR camera ❌ FAIL (ir_spoof)
Phone screen displaying face photo ❌ FAIL (ir_spoof)
No face in IR frame ❌ FAIL (no detection)

…ation

The previous IR anti-spoofing path only verified that a face-shaped
bounding box was visible in the IR frame — a printed photo or screen
replay could pass the check unobstructed.

This commit replaces that with a two-stage IR liveness pipeline:

1. YOLO face detection  — same as before, locates the face bounding box.
2. MobileNetV3 classification — crops the detected face and runs it
   through mobilenetv3_antispoof.onnx (index 0 = REAL, index 1 = SPOOF).
   Only a score below the configured threshold on the REAL class causes
   authentication to fail.

Changes
-------
auth/face/antispoofing/ir_camera_as.h
  - Rewrote doc-comment: removed 'presence-only' disclaimer, described
    the new two-stage pipeline.
  - Added antispoof_model_path / antispoof_threshold parameters.

auth/face/antispoofing/ir_camera_as.cc
  - Included face_as.h; instantiates FaceAntiSpoofing on the cropped IR
    face after detection.
  - Fails closed when the anti-spoofing model file is missing.
  - Saves failed IR frames to disk under the 'ir_spoof' label when debug
    mode is on.
  - Removed the TODO comment that tracked the missing liveness check.

auth/face/antispoofing/antispoof_check.cc
  - Wires antispoof_model_path and antispoof_threshold from FaceMethodConfig
    to the IR task so callers no longer need to supply them manually.
  - Added SAFETY comment documenting the raw-pointer lifetime contract for
    the ir_camera_session captured in the async lambda.

auth/face/antispoofing/face_as.cc
  - Fixed critical logic inversion: spoof_cls == 0 → spoof_cls == 1.
    The original code treated the REAL class (index 0) as a spoof, causing
    every real face to be rejected and every spoof to pass.
  - Added runtime output-shape validation (GetTensorTypeAndShapeInfo) to
    prevent silent out-of-bounds reads in argmax if the wrong model is
    accidentally loaded.
  - Moved argmax helper into an anonymous namespace to avoid ODR conflicts.
  - Added comment clarifying that the model emits post-softmax probabilities.

auth/face/models/mobilenetv3_antispoof.onnx
  - Added the MobileNetV3 anti-spoofing model (12 MB, tracked via Git LFS).
  - Input: 1×3×128×128 (IR greyscale expanded to 3 identical channels).
  - Output: [REAL, SPOOF] post-softmax probabilities.

docs/IR camera.md
  - Updated to describe the new liveness classification pipeline.
  - Removed outdated language about presence-only detection.
…guide

app/src/app/configuration/-components/face/FaceSetting.tsx
  - Threshold slider is now visible when IR camera is configured, even if
    the RGB AI model is disabled. Previously the slider only appeared when
    anti_spoofing.enable was true, so IR-only users had no way to adjust
    the liveness classifier threshold from the UI.

README.md
  - Updated Comparison table: 'IR camera' → 'IR liveness classification'
    to reflect that the IR path now runs a real classifier, not a
    presence check.
  - Added mobilenetv3-antispoof (facenox/face-antispoof-onnx) to the
    References section.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant