feat(antispoofing): replace IR presence check with real liveness classification#108
Open
Tunahanyrd wants to merge 9 commits into
Open
feat(antispoofing): replace IR presence check with real liveness classification#108Tunahanyrd wants to merge 9 commits into
Tunahanyrd wants to merge 9 commits into
Conversation
…ation
The previous IR anti-spoofing path only verified that a face-shaped
bounding box was visible in the IR frame — a printed photo or screen
replay could pass the check unobstructed.
This commit replaces that with a two-stage IR liveness pipeline:
1. YOLO face detection — same as before, locates the face bounding box.
2. MobileNetV3 classification — crops the detected face and runs it
through mobilenetv3_antispoof.onnx (index 0 = REAL, index 1 = SPOOF).
Only a score below the configured threshold on the REAL class causes
authentication to fail.
Changes
-------
auth/face/antispoofing/ir_camera_as.h
- Rewrote doc-comment: removed 'presence-only' disclaimer, described
the new two-stage pipeline.
- Added antispoof_model_path / antispoof_threshold parameters.
auth/face/antispoofing/ir_camera_as.cc
- Included face_as.h; instantiates FaceAntiSpoofing on the cropped IR
face after detection.
- Fails closed when the anti-spoofing model file is missing.
- Saves failed IR frames to disk under the 'ir_spoof' label when debug
mode is on.
- Removed the TODO comment that tracked the missing liveness check.
auth/face/antispoofing/antispoof_check.cc
- Wires antispoof_model_path and antispoof_threshold from FaceMethodConfig
to the IR task so callers no longer need to supply them manually.
- Added SAFETY comment documenting the raw-pointer lifetime contract for
the ir_camera_session captured in the async lambda.
auth/face/antispoofing/face_as.cc
- Fixed critical logic inversion: spoof_cls == 0 → spoof_cls == 1.
The original code treated the REAL class (index 0) as a spoof, causing
every real face to be rejected and every spoof to pass.
- Added runtime output-shape validation (GetTensorTypeAndShapeInfo) to
prevent silent out-of-bounds reads in argmax if the wrong model is
accidentally loaded.
- Moved argmax helper into an anonymous namespace to avoid ODR conflicts.
- Added comment clarifying that the model emits post-softmax probabilities.
auth/face/models/mobilenetv3_antispoof.onnx
- Added the MobileNetV3 anti-spoofing model (12 MB, tracked via Git LFS).
- Input: 1×3×128×128 (IR greyscale expanded to 3 identical channels).
- Output: [REAL, SPOOF] post-softmax probabilities.
docs/IR camera.md
- Updated to describe the new liveness classification pipeline.
- Removed outdated language about presence-only detection.
…guide
app/src/app/configuration/-components/face/FaceSetting.tsx
- Threshold slider is now visible when IR camera is configured, even if
the RGB AI model is disabled. Previously the slider only appeared when
anti_spoofing.enable was true, so IR-only users had no way to adjust
the liveness classifier threshold from the UI.
README.md
- Updated Comparison table: 'IR camera' → 'IR liveness classification'
to reflect that the IR path now runs a real classifier, not a
presence check.
- Added mobilenetv3-antispoof (facenox/face-antispoof-onnx) to the
References section.
…plicated grayscale
…o avoid use-after-free
… and check empty output shape
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The previous IR anti-spoofing path only verified that a face-shaped bounding box was visible in the IR frame. This meant a printed photo or a screen replay of a face would pass the IR check, since the YOLO model cannot distinguish a real face from a photo in IR.
This PR replaces that with a proper two-stage IR liveness pipeline and fixes a critical logic inversion in the shared
FaceAntiSpoofingclassifier.Changes
auth/face/antispoofing/face_as.cc— Critical logic fix + hardeningspoof_cls == 0→spoof_cls == 1. The original code treated the REAL class (index 0) as a spoof — every genuine face was rejected, every spoof attempt was accepted. This is now corrected.GetTensorTypeAndShapeInfo().GetShape()is called beforeargmax. If a model with an unexpected number of output classes is loaded, astd::runtime_erroris thrown immediately (caught by the surroundingtry/catch, returningfalse). This prevents silent out-of-bounds reads.argmaxinto anonymous namespace to avoid ODR conflicts with other translation units.auth/face/antispoofing/ir_camera_as.h/ir_camera_as.cc— Two-stage IR liveness pipelineantispoof_model_pathandantispoof_thresholdparameters tocheckAntispoofByIRCamera.FaceAntiSpoofing(MobileNetV3). The image has identical R, G, B channels replicated from the native GREY V4L2 format, which is what the model was trained on.ir_spooflabel when debug mode is on.TODOcomment that tracked the missing liveness check.auth/face/antispoofing/antispoof_check.cc— Wiringantispoof_model_pathandantispoof_thresholdfromFaceMethodConfigto the IR async task.// SAFETY:comment documenting the raw-pointer lifetime contract forICameraCaptureSession*captured in the lambda.auth/face/models/mobilenetv3_antispoof.onnx— New model1×3×128×128— RGB channels replicated from IR greyscale.[REAL, SPOOF]post-softmax probabilities.app/src/app/configuration/-components/face/FaceSetting.tsx— UI fixanti_spoofing.enablewastrue, so IR-only users had no way to adjust the liveness classifier threshold from the UI.docs/IR camera.md— DocumentationREADME.mdmobilenetv3-antispoofto the References section.How It Works
Both the IR pipeline and the RGB AI model run in parallel (
std::async). Authentication is granted only when all enabled methods pass.Testing
Verified on Fedora 64-bit with a RealSense IR camera (
/dev/video2):