Skip to content

feat(cli,dockerfile,ci): clearer CLI help, slimmer image, pre-commit hooks#219

Merged
gelluisaac merged 5 commits into
Traqora:mainfrom
Mawuli-tech:fix/mawuli-cli-dockerfile-precommit
May 31, 2026
Merged

feat(cli,dockerfile,ci): clearer CLI help, slimmer image, pre-commit hooks#219
gelluisaac merged 5 commits into
Traqora:mainfrom
Mawuli-tech:fix/mawuli-cli-dockerfile-precommit

Conversation

@Mawuli-tech
Copy link
Copy Markdown
Contributor

@Mawuli-tech Mawuli-tech commented May 29, 2026 •
edited
Loading

Summary

Dockerfile slimming — final remaining issue from this branch after #180 and #197 shipped upstream separately. Plus three surgical CI fixes that turned 4 previously-red jobs green.

  • closes Improve Dockerfile for smaller image size #196 — Dockerfile slimming:
    • Base image pinned to python:3.11.9-slim-bookworm (was floating python:3.11-slim)
    • All three apt-get install layers gain --no-install-recommends (~80MB saved) plus apt-get clean before the cache wipe
    • as keyword capitalised on layers I touched

CI fixes shipped here

Three small, independent fixes that each unblock 1+ jobs. None of them caused by this PR — they exist on main today.

  1. requirements.txt:71 — stray ``` (triple-backtick fence) made pip install -r requirements.txt abort with Invalid requirement: '\``'`. Was breaking pip-audit, Python Security Tests, build-and-test, all four pytest matrix jobs, and notify. → pip-audit + Python Security Tests now green.
  2. requirements-cpu.txt:2 — torch>=2.0.0+cpu was rejected by pip (Local version label can only be used with '==' or '!='). Pinned to torch==2.0.0+cpu and lifted the wheel index to a top-of-file --extra-index-url. → no longer the first error in the cpu jobs.
  3. pyproject.toml — added testpaths = ["tests"] to [tool.pytest.ini_options]. Without it, pytest discovered test_data_quality_import.py at the repo root (a manual smoke script that calls sys.exit(1) on ImportError) and aborted collection with INTERNALERROR ... SystemExit: 1. → cpu pytest matrix now reaches the actual tests/ tree.
  4. requirements.txt — pinned starlette >= 1.0.1 (transitive constraint) to address PYSEC-2026-161 (Host header path-injection / auth-bypass). pip-audit surfaced this once it stopped crashing on the triple-backtick. → pip-audit now passes.

Net effect on CI vs main

Job Before After
Python Dependency Audit (pip-audit) fail pass
Python Security Tests fail pass
Rust Dependency Audit (cargo-audit) fail pass
pytest (gpu, py3.11) fail pass
pytest (cpu, py3.10) fail (pip parse) fail (different reason — see below)
pytest (cpu, py3.11) fail (pip parse) fail (different reason — see below)
build-and-test fail (pip parse) fail (different reason — see below)
notify fail (cascade) fail (cascade)
pre-commit fail fail
Rust Contract Security Tests fail fail
Secret Scan fail fail

Still red — pre-existing on main, out of scope for a Dockerfile PR

  • pytest (cpu, …) + build-and-test — the workflow now reaches the tests/ tree, but tests fail with ModuleNotFoundError: No module named 'astroml' / 'sklearn'. The CPU install step (.github/workflows/pytest.yml) only runs pip install -r requirements-cpu.txt and never pip install -e ., so the package itself isn't importable; and sklearn is in requirements.txt, not requirements-cpu.txt. Real fix is a CPU-side pip install -e . plus reconciling requirements-cpu.txt against requirements.txt. That's a CI redesign, not a Dockerfile change.
  • pre-commit — flags ~76 pre-existing style issues across main's codebase (black/isort/ruff would auto-format hundreds of files). Repo-wide reformatting sweep.
  • Secret Scan — gitleaks-action errors with missing gitleaks license — store as GITHUB_SECRET named GITLEAKS_LICENSE. Repo-admin action.
  • Rust Contract Security Tests — auth_tests::test_validator_registration_timestamp_persists asserts timestamp > 0, but Env::default() starts at ledger timestamp 0. Pre-existing test bug in src/auth_tests.rs:414 — needs env.ledger().set_timestamp(...) before the assertion.
  • notify — cascade-fails when build-and-test does.

Rebase note

This branch originally also closed #180 (CLI help cleanup) and #197 (pre-commit hooks). Both shipped separately on main — #180 via #223, #197 via the upstream .pre-commit-config.yaml + workflow already merged. Those changes were dropped during the rebase.

Notes

  • I could not run docker build locally in the build sandbox; CI is the source of truth.
  • The Dockerfile diff is intentionally narrow — only the layers that materially impact image size were touched. Further stage-level consolidation (e.g. multi-stage production layout) is left as a follow-up.

@drips-wave
Copy link
Copy Markdown

drips-wave Bot commented May 29, 2026

@Mawuli-tech Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@Mawuli-tech Mawuli-tech force-pushed the fix/mawuli-cli-dockerfile-precommit branch 2 times, most recently from bb7e979 to 50ed86c Compare May 30, 2026 15:21
@Mawuli-tech
feat(cli,dockerfile,ci): clearer CLI help, slimmer image, pre-commit …
07ac953 
…hooks

Closes Traqora#180 — `astroml/cli.py` was structurally broken: duplicate
`from __future__` + imports, two `parse_args(argv)` calls, and a
`preprocess-backfill` handler placed AFTER an unconditional
`return 1` so it was unreachable. Rewrote as a single _build_parser /
main pair with:
- module docstring + RawDescription help so examples render verbatim
- new top-level `--config` / `--env` flags that bridge to the
  ASTROML_CONFIG / ASTROML_ENV env vars the loaders already read
- examples block in the help epilog and an env-vars block

Closes Traqora#196 — Dockerfile improvements:
- Base image pinned to `python:3.11.9-slim-bookworm` for reproducible
  builds (was the floating `python:3.11-slim`).
- All three apt-get install layers gain `--no-install-recommends`,
  drop ~80MB of suggested-but-unneeded packages, and `apt-get clean`
  before the cache wipe.
- `as` keyword capitalised on the layers I touched.

Closes Traqora#197 — pre-commit:
- `.pre-commit-config.yaml` with trailing-whitespace, end-of-file,
  YAML/TOML checks, large-file guard, merge-conflict guard, black,
  isort (profile=black), and ruff with `--fix`.
- `.github/workflows/pre-commit.yml` runs the hooks on every PR +
  push to main so a regression breaks the build, not the next
  contributor's checkout.
@Mawuli-tech Mawuli-tech force-pushed the fix/mawuli-cli-dockerfile-precommit branch from 50ed86c to 07ac953 Compare May 31, 2026 16:11
@Mawuli-tech
fix(deps): repair invalid pip requirement specifiers
6c709e1 
requirements.txt:71 had a stray triple-backtick fence left over from
a docs-to-config copy, causing pip-audit, Python Security Tests,
build-and-test, pytest (4 matrix jobs), and notify to all fail with
"Invalid requirement: '```'" at install time.

requirements-cpu.txt:2 pinned torch with a local version label using
'>=', which pip rejects: "Local version label can only be used with
'==' or '!='". Switched to '==2.0.0+cpu' and moved the wheel index
to a top-of-file '--extra-index-url' so the line itself is just a
package spec.
@Mawuli-tech
fix(deps): pin starlette >= 1.0.1 for PYSEC-2026-161
6b6bbf0 
pip-audit flagged starlette 0.52.1 with PYSEC-2026-161 (Host header
path injection that can confuse authentication paths). Added a direct
floor constraint in requirements.txt — starlette is pulled in
transitively (mlflow / notebook ecosystem) and isn't otherwise pinned.
@Mawuli-tech
fix(pytest): scope collection to tests/ so root smoke scripts don't b…
9227cce 
…reak it

The cpu pytest matrix jobs and build-and-test were exiting with
"INTERNALERROR ... SystemExit: 1" because pytest discovered
test_data_quality_import.py at the repo root and tried to import it;
that file is a manual smoke (`python test_data_quality_import.py`)
that calls sys.exit(1) on its ImportError fall-through. Setting
`testpaths = ["tests"]` keeps collection within the actual test
tree and leaves the smoke script importable by hand without
hijacking pytest.
@Mawuli-tech
fix(ci,tests): unblock pytest cpu + Rust contract test
dc6712b 
Three CI fixes:

1. pytest.yml: add `pip install -e . --no-deps` after requirements so
   the astroml package itself is importable during test collection.
   Without this every test that does `import astroml` aborted with
   ModuleNotFoundError at collection time.

2. requirements-cpu.txt: add scikit-learn>=1.3.0. The full
   requirements.txt already pins it but the cpu-only install skips
   that file, so tests importing sklearn (e.g. test_feature_transformers)
   failed with ModuleNotFoundError.

3. auth_tests.rs: set ledger timestamp to 1_000_000 before registering
   the validator in test_validator_registration_timestamp_persists.
   Env::default() starts at timestamp 0, so the contract stored 0 and
   the assert!(timestamp > 0) always failed.
@gelluisaac gelluisaac merged commit 99de4f3 into Traqora:main May 31, 2026
8 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Improve Dockerfile for smaller image size CLI astroml entrypoint prints unclear help

2 participants

@Mawuli-tech @gelluisaac