CyberSleuth: Threat Detection System

CyberSleuth is a professional-grade, Python-based security platform that integrates real-time network traffic analysis, machine learning-driven phishing detection, and automated web vulnerability scanning.

---

Key Features

• Real-Time Packet Visualization: Live traffic capture using Scapy with instant WebSocket updates to the dashboard.
• AI-Powered Anomaly Detection: Utilizes Random Forest and XGBoost models to identify suspicious network behavior and evaluate URL phishing risks.
• Vulnerability Scanning: Automated tools for scheduling and executing web scans to identify security loopholes.
• Interactive Analytics: A responsive HTML/CSS/JS dashboard featuring protocol distribution charts and detailed packet inspection.
• Session Management: Start, stop, and export network capture sessions as PCAP or JSON for forensic analysis.
• Secure Storage: All scan results and session data are logged into a local SQLite database for historical tracking.

---

Tech Stack

Component	Technology
Backend	Python, Flask
Networking	Scapy, WebSockets
Machine Learning	Scikit-learn, XGBoost, Pickle
Database	SQLite
Frontend	HTML5, CSS3 (Dark Theme), Vanilla JavaScript

---

Project Structure

.
├── core/               # Core logic for packet sniffing & scanning
├── models/             # Trained ML models (.pkl & .xml)
├── website/            # Flask templates and static assets (CSS/JS)
├── app.py              # Main Flask application entry point
├── requirements.txt    # Project dependencies
└── README.md           # Documentation

---

Installation & Setup

1. Clone repository and navigate to project root: powershell git clone <repo-url> "Cybersleuth threat detection system" cd "Cybersleuth threat detection system"
2. Create a virtual environment and activate it: powershell python -m venv .venv .\.venv\Scripts\Activate.ps1
3. Install dependencies: powershell pip install -r requirements.txt
4. Place ML models in models/random_forest_model.pkl and models/scaler.pkl. Phishing model files should reside in models/ as well.
5. Run the application: powershell python app.py The server starts on http://0.0.0.0:5000 by default.

---

API Endpoints

• Session Management
  • POST /api/sessions - create a new capture session.
  • POST /api/sessions//start - begin sniffing (loads ML model lazily).
  • POST /api/sessions//stop - stop capture.
  • GET /api/sessions//export.pcap - download the PCAP file.
• Security Services
  • POST /api/analyze/phishing - request phishing analysis for a URL.
  • POST /api/scan/web - initiate a web vulnerability scan.
• Scheduling (optional)
  • POST /api/vuln/schedule - schedule periodic vulnerability scans (requires apscheduler).
• WebSocket (/ws) pushes real-time messages of type packet, stats, phishing, and �ulnerability.

---

Using the Dashboard

1. Open http://localhost:5000/dashboard after starting the app.
2. Create a new session and click Start to begin real-time packet capture.
3. Watch the packet list populate; click entries for detailed headers and payloads.
4. Apply filters by protocol or IP, and monitor statistics at the bottom panel.
5. Use the Export PCAP button to download session data for offline analysis.

You can also interact with the backend directly through the API or use WebSocket messages for automated tooling.

---

Development & Extension

• Modify frontend templates under website/templates or static assets in website/static.
• Train new machine learning models and replace the pickled files in models/.
• Extend core/phishing_detector.py, core/network_analysis.py, or other modules to add functionality.
• The database (core/database.py) is SQLite-based; swap engines or add tables as needed.

---

Research Materials

Additional analyses and diagrams are stored in the research/ directory, which contains case studies, correlation flow diagrams, and implementation notes.

---

License

Distributed under the MIT License. See LICENSE for details.

---

Happy hunting.