From 29cce11b27844492a6837812a2ae6583123c4527 Mon Sep 17 00:00:00 2001 From: manavgup Date: Mon, 6 Apr 2026 15:20:37 -0400 Subject: [PATCH] Add developer documentation: DEVELOPING.md, TESTING.md, CONTRIBUTING.md - DEVELOPING.md: setup, project structure, how code generation works, CLI commands, file ownership model, key design decisions - TESTING.md: test organization, markers, coverage, patterns for async tests, fake repos, CLI tests, template tests, generated project tests - CONTRIBUTING.md: workflow, code standards, CI checks, commit/PR conventions, architecture decisions - CLAUDE.md: update common commands and add references to new docs Closes #14 Co-Authored-By: Claude Opus 4.6 (1M context) --- CLAUDE.md | 16 ++++-- CONTRIBUTING.md | 99 ++++++++++++++++++++++++++++++++++ DEVELOPING.md | 127 +++++++++++++++++++++++++++++++++++++++++++ TESTING.md | 140 ++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 377 insertions(+), 5 deletions(-) create mode 100644 CONTRIBUTING.md create mode 100644 DEVELOPING.md create mode 100644 TESTING.md diff --git a/CLAUDE.md b/CLAUDE.md index 61bcbde..a392c9a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -59,20 +59,26 @@ faststack/ ## Common Commands +Run `make` or `make help` for all targets. Key ones: + ```bash -make install # poetry install -make test # pytest -make test-verbose # pytest -v +make install-dev # venv + all deps + pre-commit hooks +make check # lint + typecheck + test with 85% coverage (CI gate) +make test # pytest with coverage +make test-unit # core + template tests only +make test-integration # CLI command tests make lint # ruff check + black --check make format # ruff fix + black make typecheck # mypy faststack_core/ cli/ -make check # lint + typecheck + test (CI gate) +make pre-commit # run all pre-commit hooks make clean # remove caches, build artifacts -make help # list all targets ``` ## Key Documents +- `DEVELOPING.md` — development setup, project structure, how generation works +- `TESTING.md` — test organization, markers, coverage, patterns +- `CONTRIBUTING.md` — workflow, code standards, PR process - `docs/design/fastapi-generator-plan.md` — design blueprint - `docs/implementation/v1-implementation-plan.md` — build sequence - `docs/architecture/adr/` — Architecture Decision Records (the reasoning) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..6964691 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,99 @@ +# Contributing to FastStack + +## Getting Started + +```bash +git clone https://github.com/manavgup/faststack.git +cd faststack +make install-dev # venv + deps + pre-commit hooks +make check # verify everything works +``` + +See [DEVELOPING.md](DEVELOPING.md) for project structure and how code generation works. +See [TESTING.md](TESTING.md) for test organization, markers, and patterns. + +## Development Workflow + +1. **Create an issue** before starting work on features or non-trivial fixes +2. **Create a branch** from `main`: + ```bash + git checkout -b feature/short-description + ``` +3. **Make your changes** — write code, add tests +4. **Run the full check suite:** + ```bash + make check # lint + typecheck + tests with 85% coverage + ``` +5. **Commit** with a clear message: + ```bash + git commit -m "Add foo to bar for baz" + ``` +6. **Push and create a PR** against `main` + +## Code Standards + +### Python + +- **Python 3.12+** with type hints +- **Line length:** 120 characters (ruff + black) +- **Formatting:** `make format` (ruff --fix + black) +- **Linting:** `make lint` (ruff + black --check) +- **Type checking:** `make typecheck` (mypy) + +### Naming + +- `snake_case` for functions, variables, file names +- `PascalCase` for classes +- `UPPER_CASE` for constants + +### Tests + +- Every feature or bug fix needs tests +- Unit tests for logic, integration tests for CLI commands +- Use fake repositories (Protocol-based), not mocks +- See [TESTING.md](TESTING.md) for patterns + +### Templates + +When modifying Jinja2 templates in `templates/simple/`: + +- Verify rendered output is valid Python: `ast.parse(output)` +- Test with multiple entity types (simple, with FKs, with enums, self-referential) +- Remember the REGENERATABLE vs PRESERVED distinction + +## CI Checks + +All PRs must pass these required checks before merging: + +| Check | Workflow | What it runs | +|-------|----------|--------------| +| Ruff & Black | `lint.yml` | `ruff check .` + `black --check .` | +| Mypy | `lint.yml` | `mypy faststack_core/ cli/` | +| Test (py3.12) | `ci.yml` | `pytest` with 85% coverage threshold | +| Pre-commit | `pre-commit.yml` | All pre-commit hooks | +| Package Build | `ci.yml` | `poetry build` + `twine check` | + +Run `make check` locally before pushing — it mirrors the CI gate. + +## Commit Messages + +Write clear, imperative commit messages: + +``` +Add entity list command with staleness detection +Fix pluralization for already-plural entity names +Update router template to wire Depends() injection +``` + +Lead with what the commit does, not what you did. One sentence is usually enough. Add a body only if the "why" isn't obvious. + +## Pull Requests + +- Keep PRs focused — one feature or fix per PR +- Title: short (under 70 chars), imperative +- Description: what changed and why, not a line-by-line diff +- Link the issue: `Closes #123` + +## Architecture Decisions + +Non-trivial design decisions are tracked in `docs/architecture/adr/`. If your change introduces a new pattern or overrides an existing decision, add or update an ADR. diff --git a/DEVELOPING.md b/DEVELOPING.md new file mode 100644 index 0000000..7481499 --- /dev/null +++ b/DEVELOPING.md @@ -0,0 +1,127 @@ +# Developing FastStack + +This guide gets you from zero to a working development environment. + +## Prerequisites + +- **Python 3.12+** +- **Poetry** — `pipx install poetry` +- **Git** + +## Quick Start + +```bash +git clone https://github.com/manavgup/faststack.git +cd faststack +make install-dev +make check +``` + +`make install-dev` creates a virtual environment, installs all dependencies, and sets up pre-commit hooks. `make check` runs lint + typecheck + tests with coverage — the same gate that CI runs. + +## Project Structure + +``` +faststack/ +├── faststack_core/ # Runtime library (users import this) +│ ├── base/ # Entity bases, Repository Protocol, CrudService +│ ├── exceptions/ # DomainError hierarchy + RFC 7807 handlers +│ ├── database/ # Async session config, get_db dependency +│ ├── logging/ # Structured JSON logger, sensitive data masking +│ ├── middleware/ # Correlation ID, request logging, security headers +│ ├── health/ # Health check endpoints +│ ├── settings/ # FastStackConfig dataclass +│ └── setup.py # One-call setup_app() for all middleware +├── cli/ # CLI tool (faststack command, built with Click) +│ ├── cmd_init.py # faststack init +│ ├── cmd_add_entity.py # faststack add-entity +│ ├── cmd_generate.py # faststack generate +│ ├── cmd_list.py # faststack list +│ ├── cmd_migrate.py # faststack migrate +│ ├── yaml_parser.py # YAML entity definitions → EntityDefinition +│ ├── model_introspector.py # AST-based SQLAlchemy model reader +│ └── field_mappings.py # YAML ↔ SQLAlchemy ↔ Pydantic type mappings +├── templates/ # Jinja2 templates for code generation +│ ├── project/ # Project scaffold (8 templates) +│ └── simple/ # Entity templates (11 templates) +├── tests/ # Framework tests +│ ├── test_core/ # Runtime library tests (unit) +│ ├── test_cli/ # CLI command tests (integration) +│ ├── test_templates/ # Template rendering tests (unit) +│ └── test_e2e/ # End-to-end scaffold validation +├── examples/ # Example YAML files + smoke tests +└── docs/ # Design docs, ADRs, implementation plan +``` + +## Make Targets + +Run `make` or `make help` to see all available targets, organized by category: + +- **🌱 Installation** — `venv`, `install`, `install-dev`, `update` +- **🧪 Testing** — `test`, `test-unit`, `test-integration`, `test-e2e`, `coverage` +- **🔍 Quality** — `lint`, `format`, `typecheck`, `check`, `pre-commit` +- **🧹 Cleanup** — `clean`, `clean-all` + +## How Code Generation Works + +FastStack generates projects from YAML entity definitions. The pipeline: + +``` +entities.yaml → yaml_parser.py → EntityDefinition → Jinja2 templates → .py files +``` + +1. **YAML parsing** (`cli/yaml_parser.py`) — reads entity fields, resolves FK relationships +2. **Template rendering** (`templates/simple/*.j2`) — generates 9 files per entity: + - `model.py` — SQLAlchemy ORM model + - `schema.py` — Pydantic Create/Update/Response schemas + - `repository.py` — SqlAlchemyRepository subclass + - `service.py` — CrudService subclass with lifecycle hooks + - `router.py` — FastAPI router with CRUD endpoints + Depends() wiring + - `factory.py` — Polyfactory test data factory + - `fake_repository.py` — In-memory repository for unit tests + - `test_unit_service.py` — Service unit tests + - `test_integration.py` — API integration tests +3. **Registry files** (multi-entity) — generated after all entities: + - `dependencies.py` — DI providers for all entities + - `tests/integration/conftest.py` — AsyncClient fixture with fake repo overrides + +### File Ownership + +Templates are classified as **REGENERATABLE** or **PRESERVED**: + +| REGENERATABLE (safe to overwrite) | PRESERVED (user owns) | +|---|---| +| schemas, fakes, factories | models, repos, services, routers, tests | +| `dependencies.py`, integration conftest | — | + +`faststack generate` only overwrites REGENERATABLE files. Use `--force` for PRESERVED files. + +### Model Introspection + +`faststack generate` reads existing SQLAlchemy models via AST parsing (`cli/model_introspector.py`), extracts an `EntityDefinition`, and regenerates derived files. The model is the source of truth — not YAML. + +## CLI Commands + +```bash +faststack init [--entities entities.yaml] # Scaffold project +faststack add-entity --fields "name:string:required" # Add entity +faststack add-entity --from-yaml entities.yaml # Add from YAML +faststack generate # Regenerate derived files +faststack generate --all # Regenerate all entities +faststack list # Show entity status +faststack migrate generate "message" # Create Alembic migration +faststack migrate upgrade # Apply migrations +faststack migrate downgrade # Rollback one migration +``` + +## Key Design Decisions + +See `docs/architecture/adr/` for full rationale. Summary: + +- **Async-first** — no sync support (ADR-001) +- **Protocol-based repos** — structural typing, in-memory fakes for tests (ADR-002) +- **YAML input, model source of truth** — AST introspection for regeneration (ADR-003) +- **RFC 7807 errors** — standardized error responses (ADR-004) +- **Lifecycle hooks** — `before_create`, `after_create`, etc. (ADR-005) +- **File ownership** — REGENERATABLE vs PRESERVED (ADR-006) +- **v1 = simple mode only** — DDD deferred to v2 (ADR-007) diff --git a/TESTING.md b/TESTING.md new file mode 100644 index 0000000..7e43179 --- /dev/null +++ b/TESTING.md @@ -0,0 +1,140 @@ +# Testing FastStack + +## Running Tests + +```bash +make test # All tests + 85% coverage gate (CI gate) +make test-unit # Core + template tests only (fast) +make test-integration # CLI command tests (scaffold real projects in tmp dirs) +make test-e2e # End-to-end project scaffold validation +make test-fast # Everything except @pytest.mark.slow +make test-single K=name # Run tests matching a keyword +make coverage # Generate HTML + XML coverage report +``` + +## Test Organization + +``` +tests/ +├── test_core/ # Unit tests — runtime library +│ ├── test_base_entity.py +│ ├── test_repository.py +│ ├── test_crud_service.py +│ ├── test_exceptions.py +│ ├── test_logging.py +│ ├── test_middleware.py +│ ├── test_health.py +│ └── test_setup.py +├── test_cli/ # Integration tests — CLI commands +│ ├── test_init.py +│ ├── test_add_entity.py +│ ├── test_generate.py +│ ├── test_list.py +│ ├── test_migrate.py +│ ├── test_yaml_parser.py +│ ├── test_model_introspector.py +│ └── test_field_mappings.py +├── test_templates/ # Unit tests — template rendering +│ ├── test_simple_mode.py +│ └── test_project_templates.py +└── test_e2e/ # End-to-end — scaffold + validate + └── test_smoke.py +``` + +## Markers + +Tests are auto-tagged by directory via `conftest.py` hooks: + +| Marker | Directory | What it covers | +|--------|-----------|----------------| +| `unit` | `test_core/`, `test_templates/` | Runtime library, template rendering | +| `integration` | `test_cli/` | CLI commands (scaffolds real projects in tmp dirs) | +| `e2e` | `test_e2e/` | Full project scaffold + validation | +| `slow` | (manual) | Tests taking >1s | + +Use markers for selective execution: + +```bash +poetry run pytest -m unit # Only unit tests +poetry run pytest -m "not slow" # Skip slow tests +``` + +## Coverage + +- **Threshold:** 85% (enforced in `make test` and CI) +- **Report:** `make coverage` generates `htmlcov/index.html` +- **Config:** `[tool.coverage.*]` sections in `pyproject.toml` + +Current coverage: ~90% across `faststack_core/` and `cli/`. + +## Testing Patterns + +### Async Tests + +All tests use `asyncio_mode = "auto"` (set in `pyproject.toml`). No need for `@pytest.mark.asyncio` — just write `async def test_*`. + +### Fake Repositories + +Core tests use in-memory fake repositories that satisfy the `Repository` Protocol via structural typing. No mocks. + +```python +from tests.unit.fakes.user_repository import FakeUserRepository + +@pytest.fixture +def repo(): + return FakeUserRepository() + +@pytest.fixture +def service(repo): + return UserService(repo) +``` + +### CLI Tests + +CLI tests use Click's `CliRunner` with `monkeypatch.chdir(tmp_path)` to scaffold real projects in temporary directories: + +```python +def test_init_creates_project(runner, tmp_path, monkeypatch): + monkeypatch.chdir(tmp_path) + result = runner.invoke(cli_group, ["init", "my-app"]) + assert result.exit_code == 0 + assert (tmp_path / "my-app" / "app" / "main.py").is_file() +``` + +### Template Tests + +Template tests render Jinja2 templates with `EntityDefinition` objects and verify the output is valid Python via `ast.parse()`: + +```python +def test_model_renders_valid_python(jinja_env, entity): + output = jinja_env.get_template("model.py.j2").render(entity=entity) + ast.parse(output) # Raises SyntaxError if invalid +``` + +## Testing Generated Projects + +After scaffolding a project, you can run its generated tests: + +```bash +# Scaffold +poetry run faststack init my-project --entities examples/rag_modulo.yaml + +# Run generated unit tests (100 tests for 20 entities) +PYTHONPATH=$(pwd):$(pwd)/my-project poetry run pytest my-project/tests/unit/ + +# Run generated integration tests (60 tests for 20 entities) +PYTHONPATH=$(pwd):$(pwd)/my-project poetry run pytest my-project/tests/integration/ +``` + +The `PYTHONPATH` workaround is needed because `faststack` isn't published to PyPI yet. In a real project, `poetry install` inside the generated project would handle this. + +## Pre-commit Hooks + +Pre-commit runs automatically on `git commit`. To run manually: + +```bash +make pre-commit # Run all hooks on all files +make pre-commit-install # (Re)install hooks into .git/hooks +``` + +Hooks: trailing whitespace, end-of-file, YAML/TOML validation, Python AST check, debug statement detection, private key detection, ruff lint, black formatting.