Skip to content

QUALITY_CHECKS.md

Latest commit

 

History

History
353 lines (235 loc) · 7.33 KB

QUALITY_CHECKS.md

File metadata and controls

353 lines (235 loc) · 7.33 KB

Quality Checks Guide

This guide explains how to use the quality check tools to ensure code quality before committing changes.

Overview

The RAG system includes comprehensive quality checks to maintain high code standards:

  • Code Formatting & Linting: Automatic code formatting and linting with Ruff
  • Type Checking: Static type analysis with Pyright
  • Security: Security vulnerability scanning with Bandit
  • Testing: Unit tests with pytest and coverage reporting
  • Git Status: Check for uncommitted changes

Quick Start

Run All Checks (Recommended Before Commit)

# On Windows
pre-commit-check.bat

# On Linux/Mac or using Python directly
python pre-commit-check.py

Run Checks and Auto-Fix Issues

python pre-commit-check.py --fix

Run Checks Without Tests (Faster)

python pre-commit-check.py --skip-tests

Show Detailed Output

python pre-commit-check.py --verbose

Using Make Commands

If you have make installed, you can use convenient shortcuts:

# Run all quality checks
make check

# Run checks and auto-fix issues
make check-fix

# Run checks without tests (faster)
make check-fast

# Run only tests
make test

# Run tests with coverage
make coverage

# Format code
make format

# Check formatting without changes
make format-check

# Run linting
make lint

# Run type checking
make type-check

# Run security checks
make security

# Clean build artifacts
make clean

Individual Tools

1. Code Formatting and Linting (Ruff)

Ruff is an extremely fast Python linter and formatter that replaces multiple tools (Black, isort, flake8, pylint).

# Check formatting
python -m ruff format --check rag_system tests

# Auto-format code
python -m ruff format rag_system tests

# Check linting
python -m ruff check rag_system tests

# Auto-fix linting issues
python -m ruff check --fix rag_system tests

Configuration: See pyproject.toml under [tool.ruff]

Features:

  • Combines formatting (like Black)
  • Import sorting (like isort)
  • Linting (like flake8, pylint)
  • 10-100x faster than traditional tools
  • Auto-fix capabilities

2. Type Checking (Pyright)

Pyright is a fast, feature-rich type checker for Python.

python -m pyright rag_system

Configuration: See pyproject.toml under [tool.pyright]

Features:

  • Fast type checking
  • Better error messages than mypy
  • Supports latest Python features
  • IDE integration (VS Code)

3. Security Checks (Bandit)

Bandit scans for common security issues.

python -m bandit -r rag_system -ll

Configuration: See pyproject.toml under [tool.bandit]

Severity Levels:

  • -ll: Only report medium and high severity issues

4. Unit Tests (pytest)

pytest runs all unit tests with coverage reporting.

# Run all tests
python -m pytest tests/ -v

# Run with coverage
python -m pytest tests/ -v --cov=rag_system --cov-report=term-missing --cov-report=html

# Run specific test file
python -m pytest tests/test_config.py -v

# Run specific test
python -m pytest tests/test_config.py::TestRAGConfig::test_default_config -v

Configuration: See pyproject.toml under [tool.pytest.ini_options]

Coverage Requirements: Minimum 80% code coverage

Installation

Install Development Tools

# Install all development dependencies
pip install -r requirements-dev.txt

# Or install individually
pip install ruff pyright bandit pytest pytest-cov pytest-asyncio

Add to requirements-dev.txt

ruff>=0.1.0
pyright>=1.1.0
bandit>=1.7.5
pytest>=7.3.0
pytest-cov>=4.1.0
pytest-asyncio>=0.21.0

Git Integration

Pre-commit Hook

To automatically run checks before each commit, create .git/hooks/pre-commit:

#!/bin/bash
# Run quality checks before commit

echo "Running pre-commit quality checks..."
python pre-commit-check.py --skip-tests

if [ $? -ne 0 ]; then
    echo ""
    echo "Quality checks failed. Commit aborted."
    echo "Fix the issues or use 'git commit --no-verify' to skip checks."
    exit 1
fi

exit 0

Make it executable:

chmod +x .git/hooks/pre-commit

Skip Checks (Not Recommended)

If you need to commit without running checks:

git commit --no-verify -m "Your commit message"

CI/CD Integration

GitHub Actions Example

Create .github/workflows/quality-checks.yml:

name: Quality Checks

on: [push, pull_request]

jobs:
  quality:
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v3
    
    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.9'
    
    - name: Install dependencies
      run: |
        pip install -r requirements.txt
        pip install -r requirements-dev.txt
    
    - name: Run quality checks
      run: python pre-commit-check.py

Troubleshooting

Tool Not Found

If you get "Tool not found" errors, install the missing tool:

pip install ruff pyright bandit pytest pytest-cov

Type Checking Errors

If Pyright reports type errors:

  1. Check the error message carefully
  2. Add type hints where missing
  3. Use # type: ignore for false positives (with explanation)
  4. Configure typeCheckingMode in pyproject.toml if needed

Coverage Too Low

If coverage is below 80%:

  1. Add tests for uncovered code
  2. Check coverage report: htmlcov/index.html
  3. Identify missing test cases

Ruff Auto-Fix Issues

If Ruff can't auto-fix an issue:

  1. Read the error message
  2. Fix manually
  3. Run ruff check again to verify

Best Practices

  1. Run checks before committing: Always run make check or python pre-commit-check.py

  2. Fix issues incrementally: Don't accumulate technical debt

  3. Use auto-fix when possible: Run with --fix flag to automatically fix formatting and linting

  4. Write tests first: Maintain high test coverage (>80%)

  5. Review security warnings: Don't ignore bandit warnings

  6. Keep dependencies updated: Regularly update development tools

  7. Document exceptions: If you need to ignore a warning, add a comment explaining why

Why Ruff and Pyright?

Ruff Benefits

  • Speed: 10-100x faster than traditional tools
  • All-in-one: Replaces Black, isort, flake8, pylint
  • Auto-fix: Automatically fixes many issues
  • Modern: Written in Rust, actively maintained
  • Compatible: Drop-in replacement for existing tools

Pyright Benefits

  • Fast: Much faster than mypy
  • Accurate: Better type inference
  • Modern: Supports latest Python features
  • IDE Integration: Built into VS Code
  • Clear Errors: Better error messages

Configuration Files

  • pyproject.toml: Configuration for Ruff, Pyright, pytest, bandit
  • pre-commit-check.py: Main quality check script
  • Makefile: Convenient command shortcuts

Summary

The quality check system ensures:

✓ Consistent code formatting
✓ PEP 8 compliance
✓ High code quality
✓ Type safety
✓ Security best practices
✓ Comprehensive test coverage
✓ Clean git history

Run python pre-commit-check.py before every commit to maintain these standards!

Migration from Old Tools

If you're migrating from the old setup:

  • Black + isort → Ruff format (formatting)
  • flake8 + pylint → Ruff check (linting)
  • mypy → Pyright (type checking)

Ruff and Pyright are faster, more modern, and easier to configure!