NHAI Face Auth

Offline-first biometric attendance for NHAI highway worksites
National Highways Authority of India · Hackathon 7.0 · Datalake 3.0 Ready

🌐 Full Demo Page  ·  📥 Download APK  ·  🧠 Training Notebook  ·  📊 Model Details  ·  ✨ All Features

---

📱 The App

Command Centre	Face Enrolment	Analytics Dashboard
Live KPIs · Quick actions	CNN embed · BioHash protect	Security metrics · 7-day trend

Attendance	Calendar	Worker Registry
Check-in / check-out	Monthly attendance view	Enrolled worker roster

Auth History	Admin Console	System Settings
Filterable auth log	2FA protected controls	Full configuration

---

🛣️ The Problem

NHAI manages 50,000+ construction workers across remote highway stretches with little or no connectivity. Marking attendance reliably — and proving who was actually on-site — is hard:

• 📋 Paper registers are tampered with — proxy punching is widespread
• 🌐 Cloud biometric APIs are useless without a signal
• 📍 No way to prove a worker is physically at the correct site
• ⛑️ No enforcement of safety gear (helmet, hi-vis vest) at entry

💡 Our Solution

A fully offline Android app that solves all four problems at once:

Face detected → 3 liveness challenges → anti-spoof check → CNN match (99.28% LFW)
     → GPS geofence check → PPE compliance check → attendance logged → sync to Datalake 3.0

Everything runs on-device. No server. No internet. No cloud. Works in the most remote highway stretch in India.

---

🧠 The Custom Model — 99.28% LFW

We did not use an off-the-shelf API. We trained MobileFaceNet + ArcFace from scratch.

Metric	Our Result	Hackathon Constraint	Verdict
LFW 10-fold Accuracy	99.28%	> 95%	✅ +4.28% above requirement
Model Size (INT8)	1.15 MB	< 20 MB	✅ 17× smaller than limit
CPU Latency/Face	63 ms	< 1000 ms	✅ 16× faster than limit
Self-trained	MobileFaceNet + ArcFace	Required	✅ 100% custom

Training Details

Setting	Value
Dataset	CASIA-WebFace — 490,623 images / 10,572 identities
Backbone	MobileFaceNet (~1.0 M params, 128-D embedding)
Loss	ArcFace (s=64, m=0.50)
Optimizer	SGD momentum=0.9, wd=5e-4 + warmup + cosine LR
Precision	AMP mixed fp16
Hardware	Kaggle Tesla T4 GPU
Best checkpoint	Epoch 36 — 99.28% LFW
Training time	~6.6 hours
Export	PyTorch → ONNX FP32 → dynamic INT8 (parity-checked)

📈 Full training curve (click to expand)

Epoch	LFW Acc		Epoch	LFW Acc
1	85.90%		20	98.18%
2	94.47%		24	98.57%
4	96.43%		28	98.83%
7	97.22%		31	99.07%
10	97.78%		33	99.13%
12	98.07%		36	99.28% ★ BEST
17	98.47%		40	99.15%

On-device resilience: the app runs the ONNX FP32 model with an eye-aligned geometric landmark fallback — recognition works on every Android phone, even those whose ONNX runtime lacks the quantized operator set.

📓 Reproduce it yourself: FaceAuthApp/notebook/mobilefacenet_training.ipynb — open on Kaggle, attach CASIA-WebFace, set GPU T4, Run All.

📦 Trained weights: FaceAuthApp/artifacts/ — mobilefacenet_fp32.pt · mobilefacenet_fp32.onnx · mobilefacenet_int8.onnx

---

✨ Features

🧠 Face Recognition & Matching

• Custom MobileFaceNet CNN — 128-D L2-normalised embeddings, 99.28% LFW
• Eye-aligned ArcFace 112×112 similarity transform (matches training exactly)
• Cosine similarity matching with method-adaptive thresholds (CNN: 0.42, geometric: 0.80)
• Duplicate enrolment detection — blocks re-registration of the same person
• Geometric landmark fallback — eye-aligned, pose/scale invariant, runs on every device

👁️ Liveness + Anti-Spoofing (3-layer)

• Layer 1: Active liveness — 3 randomized challenges (blink / smile / head-turn)
  • Baseline-relative head-turn detection — works at any camera angle
  • Live progress bar guides the user in real time
  • Randomized order every session — defeats pre-recorded video attacks
• Layer 2: Passive anti-spoof — Laplacian-variance texture analysis (native Kotlin, <10 ms)
  • Printed photos and phone screens blocked before matching begins
• Layer 3: Face classification quality — eye-open probability + smile probability gating

🔐 Privacy & Security

• BioHash (ISO/IEC 24745) — raw face embedding never stored, cancellable templates
• AES-256-GCM encryption at rest for all biometric data
• 3-attempt lockout + 30 s cooldown against brute-force
• GDPR-style data retention with automatic purge of expired records
• Aadhaar Verhoeff checksum validation + masked display (XXXX XXXX 1234)

📍 Field Features

• GPS Geofencing — Haversine distance vs configured site radius; check-in outside the site fails
• PPE Compliance — helmet + hi-vis vest detection gates site entry (configurable: warn or hard-block)
• Offline-first sync — all writes to encrypted local store; background push to Datalake 3.0 with exponential-backoff retry
• Live analytics — liveness pass rate, match confidence, spoof blocks, geofence compliance, 7-day trend
• Hindi / English voice prompts (TTS) + high-contrast outdoor UI

---

🏗️ Architecture

┌────────────────────────────────────────────────────────────────┐
│                   React Native 0.85  (Hermes, New Arch)         │
│   Home · Enrol · Authenticate · PPE · Dashboard · Admin · …     │
└─────────────────┬──────────────────────────────┬───────────────┘
                  │                              │
     ┌────────────▼────────────┐    ┌────────────▼──────────────┐
     │  Kotlin Native Module    │    │  TypeScript Services       │
     │  FaceProcessor           │    │  embeddingUtils (cosine)   │
     │  • ML Kit async detect   │    │  bioHash (ISO/IEC 24745)   │
     │  • MobileFaceNet FP32    │    │  encryption (AES-256-GCM)  │
     │  • Eye-aligned crop      │    │  geofencing · ppeDetection │
     │  • Laplacian anti-spoof  │    │  syncService · datalake    │
     │  • Geometric fallback    │    │  adaptiveThreshold · i18n  │
     └────────────┬────────────┘    └────────────┬──────────────┘
                  │                              │
     ┌────────────▼──────────────────────────────▼──────────────┐
     │   Encrypted local store (AsyncStorage + AES-256-GCM)      │
     │   Background sync → NHAI Datalake 3.0 API                 │
     └───────────────────────────────────────────────────────────┘

End-to-end pipeline latency: 300–500 ms (capture 50ms + detect 100ms + embed 63ms + match <10ms + record <50ms)

---

🛡️ Security — Every Attack, Every Mitigation

Threat	Attack	Mitigation	Status
Photo spoof	Printed photo to camera	Laplacian-variance texture (native Kotlin)	✅
Screen replay	Video on another phone	Laplacian flat-texture detection + liveness	✅
Pre-recorded video	Video with correct moves	Randomized 3-of-4 challenge order	✅
Proxy attendance	Person A for person B	CNN recognition 99.28% + liveness	✅
Location fraud	Remote / GPS spoof	Haversine geofence vs site boundary	✅
Duplicate enrolment	Re-register same person	Cosine similarity ≥ 0.55 rejection	✅
Device theft	Physical access	AES-256-GCM encrypted store + lockout	✅
Biometric extraction	Steal template	BioHash ISO/IEC 24745 — raw never stored	✅
Brute force	Repeated attempts	3-attempt lockout + 30 s cooldown	✅
Record tampering	Edit attendance	Server-timestamp validation + audit trail	✅

---

🆚 Why We Win

Capability	✅ NHAI Face Auth	❌ Typical Systems
Offline operation	Full offline — every feature on-device	Requires connectivity
Recognition model	Self-trained 99.28% LFW, 1.15 MB	Cloud API or 10–50 MB models
Template protection	BioHash ISO/IEC 24745 — raw never stored	Raw embedding stored
Liveness	3 randomized challenges + passive anti-spoof	Single factor or none
Device coverage	CNN + geometric fallback — every phone	Fails on unsupported ops
Safety gate	GPS geofence + PPE helmet/vest check	Attendance only
Privacy	ISO/IEC 24745, AES-256, GDPR retention	Varies / none
Identity	Aadhaar Verhoeff + masked display	Not integrated
Analytics	Live on-device dashboard — works offline	Cloud only

---

🔌 Datalake 3.0 Integration

import { FaceAuthModule } from './FaceAuthApp/src/services/datalakeIntegration';

// One call: face auth + liveness + geofence + attendance + sync
const result = await FaceAuthModule.markAttendance(imagePath);
// result.authenticated      → true / false
// result.withinGeofence     → true / false
// result.attendanceAction   → "CHECKED IN" / "CHECKED OUT"
// result.matchScore         → 0.0 – 1.0
// result.livenessPassed     → true / false
// result.bioHashVerified    → true / false

// Background sync when online
await FaceAuthModule.syncToServer();

REST API endpoints: POST /enroll · POST /auth/verify · POST /attendance/sync · GET /sites · POST /ppe/report · GET /models/latest

---

🚀 Quick Start

A) Install APK (no build needed)

# Download from GitHub Releases, then:
adb install -r NHAI-FaceAuth.apk
# Open app → Enrol New Worker → Scan & Authenticate

B) Build from source

git clone https://github.com/Eartherai/FaceAuthApp.git
cd FaceAuthApp/FaceAuthApp
npm install
cd android && ./gradlew assembleDebug
# → android/app/build/outputs/apk/debug/app-debug.apk

Prerequisites: Node ≥ 18, JDK 17, Android SDK 35 + NDK (via Android Studio)

C) Reproduce the model

1. Open FaceAuthApp/notebook/mobilefacenet_training.ipynb on Kaggle
2. Attach CASIA-WebFace .rec dataset
3. Settings → Accelerator → GPU T4
4. Run All  →  exports fp32.pt / fp32.onnx / int8.onnx + constraints check

D) Tests

cd FaceAuthApp
npx jest          # bioHash, embeddings, qualityGate, Aadhaar, retryPolicy…
npx tsc --noEmit  # TypeScript type-check

---

📂 Repository Structure

nhai-face-auth/
├── README.md                          ← you are here
├── LICENSE                            (MIT)
├── docs/                              GitHub Pages site
│   ├── index.html                     full interactive demo page
│   ├── nhai_logo.png
│   ├── screenshots/                   9 app screenshots
│   ├── MODEL.md                       model training deep-dive
│   ├── FEATURES.md                    complete feature notes
│   ├── ARCHITECTURE.md               system design
│   ├── SECURITY.md                    threat model
│   └── BUILD.md                       reproducible build guide
├── FaceAuthApp/                       React Native application
│   ├── src/
│   │   ├── screens/                   12 screens (Home, Enrol, Auth, PPE…)
│   │   └── services/                  bioHash · encryption · geofencing · sync…
│   ├── android/
│   │   └── app/src/main/java/…/
│   │       └── FaceProcessorModule.kt Kotlin: ML Kit + ONNX + anti-spoof
│   ├── notebook/
│   │   └── mobilefacenet_training.ipynb  full Kaggle training notebook
│   ├── artifacts/
│   │   ├── mobilefacenet_fp32.pt      PyTorch checkpoint
│   │   ├── mobilefacenet_fp32.onnx    FP32 ONNX (4.0 MB)
│   │   └── mobilefacenet_int8.onnx    INT8 ONNX (1.15 MB) ← used on device
│   ├── __tests__/                     unit tests
│   └── TECHNICAL_DOCUMENT.md         full technical reference
└── presentation/
    └── NHAI_FaceAuth_FINAL.pptx       hackathon pitch deck

---

🛠️ Tech Stack

Layer	Technology
Framework	React Native 0.85, New Architecture, Hermes
Camera	VisionCamera v5.0.11
Face Detection	Google ML Kit (offline, bundled)
Recognition Model	Custom MobileFaceNet + ArcFace (self-trained)
ML Runtime	ONNX Runtime Android 1.18
Native Module	Kotlin (FaceProcessorModule)
Encryption	AES-256-GCM
Storage	AsyncStorage (encrypted)
Navigation	React Navigation 7
Language	TypeScript 5.8
Training	PyTorch 2.x, ONNX, scikit-learn
Platform	Android 7.0+ (API 24), Target API 35

---

📚 Documentation

Document	Description
🌐 Full Demo Page	Interactive site: every feature, model details, architecture, comparison
📊 MODEL.md	Training setup, architecture, results, reproduction, integration
✨ FEATURES.md	Every feature, how it works, why it matters, criterion map
🏗️ ARCHITECTURE.md	Pipeline, modules, data flow, performance budget
🛡️ SECURITY.md	Full threat model, mitigations, privacy guarantees
🔨 BUILD.md	Prerequisites, clean build, run, test, troubleshooting
📄 TECHNICAL_DOCUMENT.md	Complete technical reference
🧠 Training Notebook	Kaggle notebook: train → evaluate → export

---

NHAI Face Auth — National Highways Authority of India · Hackathon 7.0

Custom MobileFaceNet (99.28% LFW) · Offline-first · Privacy by design