tech_debt_cleanup: resolve P1/P2/P3 review backlog (~145 todos) + 4 critical PR-review fixes#194
tech_debt_cleanup: resolve P1/P2/P3 review backlog (~145 todos) + 4 critical PR-review fixes#194unclesp1d3r wants to merge 57 commits into
Conversation
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
…ling Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
- #26 (P4/R-M5): Add lto="fat" and panic="abort" to [profile.release]; drop now-redundant lto="thin" from [profile.dist]. No should_panic tests. - #27 (P5): Replace per-byte format!("{:02x}") in type_transformer with static HEX_CHARS lookup + preallocated String. Output bytes unchanged. - #40 (D-H4/D-H5): Remove obsolete "exit(-1)=255" claim and rewrite the main.rs entry point section to describe CLI-first + env fallback + EXIT_CONFIG_ERROR=2 contract. - deps: aws-lc-rs 1.16.0 -> 1.16.3 (cascades aws-lc-sys 0.37.1 -> 0.40.0), clearing RUSTSEC-2026-0045/0046/0047/0048 (PKCS7 verify / CRL scope bypasses). Minor bump, no API changes. Required to make cargo deny check (part of just ci-check) pass. just ci-check: advisories ok, bans ok, licenses ok, sources ok; 479/479 tests pass. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
- #37 (D-H1): Add //! module-level docstrings to all 10 src files (lib, main, cli, exit, tls, type_transformer, json, csv, tab, utils). Each preamble summarises the module's purpose, public contract, and key invariants. Replaces a misplaced `///` doc comment at the top of utils.rs that was silently attached to a `use` statement. - deny.toml: Remove windows-sys multi-version skip (only 0.61.2 remains after recent dep updates — skip was flagging as unnecessary-skip). Clarify webpki 0.22.4 as ISC via [[licenses.clarify]] — webpki ships a LICENSE file but no `license` field in Cargo.toml, producing a no-license-field warning. LICENSE content verified (hash 0x001c7e6c). just ci-check: advisories ok, bans ok, licenses ok, sources ok; 479/479 tests pass, zero warnings. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Co-authored-by: Copilot <copilot@github.com> Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
- #9 (D-C1): Remove stale "Pattern Matching Bug" claim from AGENTS.md (Other Critical Issues section + Quick Reference table). Bug was fixed long ago; src/main.rs uses correct Some(_) pattern. - #10 (D-C2): Remove duplicate TLS-migration narrative from CHANGELOG [Unreleased] (the same content already lives in v0.2.6 - 2025-09-04). Drop the contradictory "CHANGELOG vs Cargo.toml" version-mismatch claim from AGENTS.md and the hardcoded "v0.2.6" architecture heading. - #36 (T-H9): Add three regression tests in src/main.rs::tests for the CLI-overrides-env precedence path (db_url, query, output_file). Pins the historical Some(&_) pattern bug. - #38 (D-H2): Document FormatWriter trait's actual scope in lib.rs -- only JsonWriter implements it; csv/tab use the free-function pattern. Resolution of the trait-vs-module question deferred to todo #20. - #39 (D-H3): Fix the format-module writer contract in AGENTS.md to match the actual signature (Item = String, output: W by value, not impl AsRef<str> / &mut W). Cursor rules file updated locally but is gitignored. - #41 (D-H6): Document --allow-invalid-certificate MITM exposure in SECURITY.md (new "Operational Risk Surface" section), README.md (TLS example + note under CLI options), and src/cli.rs long-help. - #42 (D-H7): Document --query-file path-read scope in the same SECURITY.md section, README.md CLI options table, and src/cli.rs long-help. Pairs with todo #23 (sandboxing). - #43 (D-H8): Add "Known Limits" section to README.md and "Memory Profile (F007)" to docs/src/usage/configuration.md with concrete rows-vs-RAM table and mitigation guidance until F007 streaming lands. - #44 (D-H9): Document --dump-config best-effort redaction caveat in SECURITY.md, configuration.md, src/cli.rs long-help, and the README example comment. Pairs with todos #004 and #29. Snapshot updates: cli_testing_example__help_output, tls_backward_compatibility__cli_help_output, and tls_cli_integration__tls_help_options now reflect the new --help prose for --query-file, --dump-config, and --allow-invalid-certificate. just ci-check: 482/482 tests pass, advisories/bans/licenses/sources all ok, zero warnings. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Adds a `GoldDiggerError` enum in `src/exit.rs` with stable
`exit_code() -> i32` mapping for the public 0-5 contract:
NoRows -> 1, Config -> 2, DbAuth -> 3, Query -> 4, Io -> 5,
Tls -> 3 (or 2 for CaFileNotFound / InvalidCaFormat /
MutuallyExclusiveFlags).
`map_error_to_exit_code` is reworked to two-pass classification:
1. Typed path (preferred): walks the anyhow error chain and
downcasts to `GoldDiggerError`, `TlsError`, or `std::io::Error`.
Returns the variant's exit code. Stable under message-text
refactors.
2. Substring fallback (legacy): the existing keyword classifier
runs only when no typed variant is found in the chain. Slated
for removal once all construction sites are migrated (tracked
under todos #17 - tls.rs 80-line classifier, #31 - test
sweep, #165 - integration coverage for codes 3 and 5).
Migrates the three resolve_* functions in main.rs to construct
`GoldDiggerError::Config` and `GoldDiggerError::Io` directly so the
typed path is exercised in production from the first error site
users hit. Existing message text is preserved (the new wrapper is
"configuration error: <original>"), so the resolver tests'
substring assertions still pass.
Adds five new tests in `src/exit::tests` that exercise the typed
path independent of message text:
- test_typed_error_exit_codes_direct
- test_typed_error_through_anyhow_chain
- test_typed_path_overrides_substring_match
- test_io_error_downcast_path
- test_no_rows_variant_overrides_default_query_classification
Existing six substring tests retained as they exercise the legacy
fallback path that still serves un-migrated call sites.
just ci-check: 487/487 tests pass, advisories/bans/licenses/sources
all ok, zero warnings.
Closes #2 (foundation). Substring fallback removal blocked on
typed migration of remaining sites (#17, #31, #165).
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
…rage Resolves P1-J batch (test hygiene + proptest, 5 todos): - #7/#32: add proptest dev-dep (1.11) and five property tests in src/exit.rs guaranteeing (a) the substring classifier returns a code in [1..=5] for any input and never panics, and (b) the typed GoldDiggerError::{Config,DbAuth,Query} variants produce a stable exit code regardless of message text, even through anyhow::Error context layers. 1024 cases per test, ~0.26s total. - #31: sweep five bare .failure() calls in tests/cli_testing_example.rs (no remaining .failure() anywhere under tests/). Each site now asserts the specific expected exit code per src/exit.rs contract: .code(2) for missing-config / Clap mutex errors, .code(3) for unreachable-host connection failures. Refactor the cli_command rstest fixture to env_remove DATABASE_URL / DATABASE_QUERY / OUTPUT_FILE / NO_COLOR. - #31: add clean_cmd() helper to tests/test_support/cli.rs and env_remove inside GoldDiggerCommand::execute() so every command built through the support module starts from a known-clean environment. - #165: new tests/exit_code_3_auth.rs asserts unreachable-host runs exit with EXIT_DB_AUTH_ERROR (3). New tests/exit_code_5_io.rs asserts chmod-0o000 query-file dir produces EXIT_IO_ERROR (5) on Unix, with a Windows fallback that points --query-file at a path whose parent is a regular file. Both use cargo::cargo_bin_cmd!(); Unix path skips when running as root. - #166: audited all 10 test files that spawn the binary. Three files (tls_cli_integration.rs, tls_backward_compatibility.rs, tests/integration/common.rs) were missing env_remove — each now has a local fresh_cmd() helper that strips the four Clap-bound env vars. The cli_testing_example fixture was patched likewise. The remaining six files (exit_codes.rs, exit_code_3_auth.rs, exit_code_5_io.rs, credential_leak_regression.rs, dump_config_redaction.rs, test_support/cli.rs) already env_removed or do so via clean_cmd. Also migrates 14 deprecated Command::cargo_bin("gold_digger") call sites to the cargo_bin_cmd! macro form (or to fresh_cmd / clean_cmd helpers that wrap it), matching commit f3021b3's typed-error preference. Verification: `just ci-check` -> 546 tests pass (17 in src::exit::tests including the five new proptests), advisories/bans/licenses/sources ok, TLS validation passes. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
…uites (P1-C) Consolidates the three divergent credential-redaction implementations (utils::redact_sql_error, tls::redact_url, main::dump_configuration inline) into a single module (src/utils.rs) so pattern fixes land everywhere at once. The previously-weakest redactor lived in the highest-risk output surface (--dump-config); it now routes through the same regex set as the mysql error path. Changes: - src/utils.rs: adds redact_url (moved from tls.rs), redact_dump_query, and shared placeholder constants. Extends pattern set with pwd=, passwd=, pass=, SET PASSWORD, IDENTIFIED WITH ... BY, Kennwort, mot_de_passe, and contraseña. Adds idempotence and non-English tests. - src/tls.rs: Pool::new map_err now funnels every mysql_error string through utils::redact_sql_error before interpolation. tls::redact_url becomes a thin re-export to utils::redact_url. Closes CWE-532/209 leak in 80-line auth/URL/handshake wrap (todo #3, S1). - src/main.rs: dump_configuration replaces the substring heuristic ('password' / 'identified by') that either missed GRANT/SET PASSWORD/pwd/passwd/base64/JWT/non-English OR obliterated the legitimate query with a sentinel. Now delegates to redact_dump_query so redaction is surgical, not nuclear (todos #004, #18). - tests/credential_leak_regression.rs: sentinel-based E2E suite asserting no password/username sentinel reaches stdout/stderr across 6+ failure surfaces (unreachable host, bad port, URL-encoded pw, malformed scheme, query-file with credentials, --dump-config with sensitive DATABASE_URL/DATABASE_QUERY). Synthesizes the server-side auth-denied string to pin the redactor contract without a live DB (todos #6, #35). - tests/dump_config_redaction.rs: rstest truth table of 15 adversarial query shapes driven through --query and DATABASE_QUERY paths; asserts identical redaction fidelity between the two, plus a pass-through test for benign queries (todo #29). Resolves todos #3, #004, #6, #16, #18, #29, #35. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
…lows Resolves the CI/devops slice of the P1 review backlog (todos #34, #48, #49, #50, #51, #52): - tests: drop `if is_ci() { return Ok(()) }` skip guards from container tests in `type_safety.rs`, `tls_integration_old.rs`, `integration/tls_tests.rs`, and `tls_variants_test.rs`. Real TLS and type-safety flows now execute on Linux GitHub Actions runners (Docker is available) instead of passing silently. Per-platform skip is delegated to the runner matrix and `is_docker_available()`. - coverage workflow: new `.github/workflows/coverage.yml` invokes `cargo llvm-cov --fail-under-lines 80` so coverage regressions block PR merges instead of degrading silently behind Codecov uploads. - cargo-audit PR gate: new `.github/workflows/cargo-audit-pr.yml` runs the `actions-rust-lang/audit` action on `pull_request` and `push` to `main`, closing the window between merge and the next daily `audit.yml` cron tick. - benchmarks workflow: new `.github/workflows/benchmarks.yml` compiles Criterion benches on PR (`--no-run`) and runs the full bench suite on `main` against a containerized `mysql:8.4` service, uploading results as artifacts so DB-backed benches (`rows_processing`, `value_conversion`) finally execute in CI. - CodeQL: new `.github/workflows/codeql.yml` enables the GitHub Rust CodeQL ruleset on PR, push to `main`, and weekly schedule. The file was already on the zizmor unpinned-uses ignore list. - dependabot: switch the github-actions ecosystem from daily/limit-10 to weekly/limit-5 with a `groups:` block that batches minor/patch bumps, and add a Cargo grouping that isolates security-sensitive crates (mysql/rustls/clap/tokio/serde patches) from the bulk `rust-deps` group. Cuts the recent 4-of-5 individual-bump commit noise without blurring security-sensitive review. Skipped from this batch: - todo #13 (SHA-pin release.yml + zizmor exemption removal) requires editing `.github/workflows/release.yml`, which is currently part of the maintainer's in-progress workflow edits. Left in place. All new workflows pin third-party actions to the full SHAs already established in `dist-workspace.toml` and the existing workflows. Local `just ci-check` passes (546/546 tests, deny + TLS validation green). Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
…P1-G) #25 (rows_to_strings drain): switch the conversion loop from `rows.iter()` to `rows.drain(..)` so each `mysql::Row` is dropped as soon as its string representation has been extracted. Peak memory during conversion now scales with the result vector alone (~1x raw payload) rather than holding the source `Vec<Row>` and the fully materialised `Vec<Vec<String>>` simultaneously (~2x). For large result sets the saving is roughly half of the source-row footprint; on the synthetic 100k row x 10 col case the resident-set delta drops into the streaming-friendly band recorded by the new benchmark. The function signature is unchanged (still `Vec<Row>` by value); the single internal binding is now `mut rows`. #33 (memory_ceiling benchmark): add `benches/memory_ceiling.rs` exercising `rows_to_strings` at 1k / 10k / 100k synthetic rows. The benchmark builds rows in-process via `mysql_common::row::new_row` (no live MySQL required) and reports RSS via `sysinfo` plus an estimated `Vec<Vec<String>>` footprint, so future F007 streaming work has a regression-guard baseline. Registered in `Cargo.toml` under `[[bench]] name = "memory_ceiling" harness = false`. Numbers, not assertions -- the benchmark is a guard, not a gate. Acceptance criteria addressed - #25: rows dropped progressively during conversion; output stable; `just ci-check` green (546 tests passing). - #33: bench compiles, runs without OOM at 100k rows, prints stable size/RSS lines for trend tracking; covered by `cargo build --benches`. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
…second-confirmation gate Resolves 7 P1 todos around TLS error classification, security hardening, and layer coupling: - #17: Replace the 80-line `to_lowercase().contains(...)` closure in `Pool::new` with a typed match on `mysql::Error` variants (`IoError`, `TlsError`, `MySqlError`, `DriverError`, `UrlError`, `CodecError`, etc.). TLS wrapper errors are unwrapped and delegated to `TlsError::from_rustls_error`; all user-facing strings still flow through `utils::redact_sql_error` (P1-C redaction closure preserved). - #12: Remove all `format!("{:?}", ...)` Debug-based classification from `from_rustls_error`. Variants of `rustls::Error`, `CertificateError`, and `PeerIncompatible` are matched directly. Debug is used ONLY for surfacing variant names in user-facing messages, never for routing. - #21: Fail-closed — unknown `CertificateError` variants default to `TlsError::CertificateValidationFailed` (not `ConnectionFailed`), so a future rustls bump that adds a cert variant never silently demotes validation failure to a retry hint (CWE-754 / CWE-392). Dead `san` substring branch removed with the closure rewrite. - #22: `--allow-invalid-certificate` now requires a second-confirmation flag `--i-understand-this-is-insecure` (or env `GOLD_DIGGER_ALLOW_INVALID=1`). Without the companion, `to_tls_config()` returns `TlsError::MutuallyExclusiveFlags` → exit 2 (CWE-295 / CWE-296). `display_security_warnings` now sleeps 3 seconds after the `[DANGER]` banner in `AcceptInvalid` mode (skipped under `cfg(test)`). - #45: Move `from_tls_options` adapter from `tls.rs` to `cli.rs` as `TlsOptions::to_tls_config`. `tls` module now depends only on primitive types (`bool`, `Option<PathBuf>`), enabling future extraction. - #28: Add `tests/tls_classifier_rcgen.rs` (14 tests) using `rcgen` to generate real PEM material and drive the classifier through the same paths the production Pool::new closure takes. Includes the fail-closed assertion for unknown cert variants and redaction-hygiene checks. - #46: Delete the 18 `TlsError::*` constructor helper methods. All callers (src + tests) migrated to direct struct init. Also updates 3 insta snapshots for the new `--i-understand-this-is-insecure` flag and propagates the companion flag through 28 integration test sites. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
The monolithic `src/tls.rs` mixed the error taxonomy, the rustls->TlsError
classifier, the config DTO, the CA-file loader, and the Pool builder in
one ~1900-line file with ~1100 lines of inline tests. Per the project's
file-size guideline (>=500 preferred, 1000 hard limit), split into a
directory module where each file has a single concern:
- src/tls/mod.rs (43 lines) - //! doc + pub re-exports
- src/tls/error.rs (487 lines) - TlsError enum + ctors + is_* helpers
- src/tls/classifier.rs (285 lines) - rustls::Error -> TlsError mapping
- src/tls/config.rs (812 lines) - TlsConfig, TlsValidationMode, adapter
- src/tls/ca.rs (124 lines) - CA PEM loading (re-exported as cert_utils)
- src/tls/pool.rs (266 lines) - create_tls_connection + redact_url
The public API is preserved verbatim via `pub use` in `mod.rs`, so the
existing `gold_digger::tls::{TlsError, TlsConfig, TlsValidationMode,
create_tls_connection, redact_url, cert_utils, tls_config_from_url}`
import paths used by src/main.rs, src/cli.rs, src/exit.rs, and the
tests/ suite continue to resolve unchanged. Tests (inline #[cfg(test)])
moved to the submodule that owns the code under test; no test coverage
was lost.
No behavioural changes:
- All 45 inline tls:: unit tests pass.
- All external tls integration tests pass (tls_classifier_rcgen,
tls_config_unit_tests, tls_backward_compatibility, tls_variants_test).
- Full `cargo nextest run` reports 552/552 passed, 13 skipped.
- `cargo clippy --all-targets -- -D warnings` is clean.
- `cargo deny check` reports advisories/bans/licenses/sources all ok.
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
…s (P1-E) Resolves 5 P1 todos in the format/output cleanup cluster. Net effect: one `json::write` entry point, one free-function contract across all three writers, and a fail-fast OutputFormat resolver that refuses to guess on unknown extensions. - #8 Delete JSON string-inference path. `JsonWriter::write_row` used to run every value through `parse::<u64>() → parse::<i64>() → parse::<f64>() → to_lowercase()=="true"` and coerce matches into JSON numbers/bools. That corrupted ZIP codes (`"00123"` → `123`), string flags (`"TRUE"` → `true`), phone numbers, country codes, and any identifier that happened to parse as a number. The typed-JSON path via `TypeTransformer::row_to_json` is now the only path. Pinned snapshots in `tests/tests/snapshots/json_null_handling.snap` migrated (NULLs are now `serde_json::Value::Null`, not `""`). Added two regression tests in `src/json.rs` that pin the new semantics: leading-zero strings and mixed-case `"TRUE"`/`"FALSE"` stay strings. - #15 Delete 3-of-4 JSON write entry points. Removed `JsonWriter` struct, `json::write()`, `json::write_with_pretty()`, and the separate `write_typed(Vec<mysql::Row>, ...)` re-export. The sole remaining entry point is `json::write(maps, output, pretty)` which mirrors `csv::write` / `tab::write` for contract symmetry. The convert-before-create-file discipline is preserved: the caller validates all rows via `TypeTransformer::row_to_json` before the output file is opened, so a type error on row N never leaves a truncated file behind. src/json.rs shrank from 461 → 176 lines. - #20 Delete the unused `FormatWriter` trait. It was implemented only by the now-deleted `JsonWriter`, its signature (`Item = &[String]`) did not match the documented `AsRef<str>` contract in AGENTS.md, and `main.rs::write_output` never dispatched through it. Picked the simpler "Option A" from the todo: single free `write()` per module, no trait abstraction. Updated AGENTS.md writer-contract paragraph to match. - #19 `OutputFormat::from_extension` returns `Option<Self>`. Previous signature silently fell back to TSV for `.xml`, `.yaml`, `.data`, uppercase `.CSV`, or a missing extension — violating fail-fast, and combining poorly with the symlink/traversal work on #23. New impl lowercases the extension for case-insensitive matching and returns `None` on unknown/missing extensions. `main.rs::write_output` now raises `GoldDiggerError::Config` with a remediation hint ("Cannot infer output format ... Pass --format <csv|json|tsv> ...") instead of silently emitting TSV. Added six unit tests in `src/cli.rs` covering lowercase, uppercase, missing, and unknown extensions. - #11 Remove the vestigial `json` and `csv` feature flags. Both were empty marker features (`json = []`, `csv = []`) in the default set; the `csv` crate was always linked, the `csv`/`json` modules were compiled unconditionally, and `--no-default-features --features "json csv"` produced a binary byte-for-byte identical to the default build. README and docs advertised a non-existent attack-surface reduction. Dropped the flag definitions from Cargo.toml, removed the `#[cfg(feature = "json")]` and `#[cfg(feature = "csv")]` markers from `src/main.rs`, `src/lib.rs`, and `dump_configuration`, and updated every occurrence of `--features "json csv additional_mysql_types verbose"` in justfile to `--features "additional_mysql_types verbose"`. Release binary size is unchanged (3,250,832 bytes) — confirming the flags gated nothing. Tests: 565/565 pass via `just ci-check`; advisories/bans/licenses/ sources OK; rustls-only dep tree preserved. Snapshot migration `tests/tests/snapshots/json_null_handling.snap` is the only touched snapshot file; it pins the new null-as-JSON-null contract. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Implements the P1-F path-safety batch (todos #23, #24, #30): - #23: validate `--query-file` before reading. The path is canonicalised via `std::fs::canonicalize` (resolves `..` and symlinks), the target extension is checked against a case-insensitive deny-list (`.exe`, `.dll`, `.so`, `.dylib`, `.bin`, `.bat`, `.cmd`, `.com` refused; `.sql`, `.txt`, or missing allowed), and files larger than 10 MiB are refused with a configuration error to cap DoS risk. Broken symlinks or missing files surface as `GoldDiggerError::Io` (exit 5); policy rejections map to `GoldDiggerError::Config` (exit 2). - #24: harden output file creation. All three `File::create` call sites in `write_output` now go through `create_output_file`, which on Unix uses `OpenOptions::create_new(true).custom_flags(libc::O_NOFOLLOW) .mode(0o600)`. Pre-existing targets are refused with a configuration error pointing at the new `--force` flag; symlinks at the target are refused regardless of `--force` (O_NOFOLLOW). `libc` is added as a unix-only direct dep for `O_NOFOLLOW`. The Windows path falls back to `create_new(true)` / `create(true) + truncate(true)` under `--force`. - #30: add `tests/path_safety.rs` with 10 tests covering the `.exe` extension reject, uppercase `.SQL` accept, missing-extension accept, size-cap reject, symlink-after-canonicalise reject, broken-symlink I/O error, pre-existing-output reject, `--force` bypass, Unix symlink output reject via O_NOFOLLOW, and `--force` on missing output path. Integration-test common helpers pass `--force` at each `--output` site because they pre-seed the path with `NamedTempFile`. SECURITY.md is updated to document the new guards for both `--query-file` and `--output`. Help-text snapshots are regenerated for the new `--force` flag and updated `--query-file` prose. `just ci-check` passes (575 tests; advisories/bans/licenses/sources ok). Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
… (P1-H) Resolve the three P1-H UX todos in a single batch so the new logging, colour, and progress code share one helper module and one set of NO_COLOR / TTY / quiet detection rules. #161 (color output) — add owo-colors with the supports-colors feature. Introduce `logging::warn_banner` and `logging::danger_banner` that wrap `[WARNING]` / `[DANGER]` text in yellow / red+bold ANSI codes when stderr is a TTY AND `NO_COLOR` is unset. TLS `display_security_warnings` now renders coloured banners for `SkipHostnameVerification` and `AcceptInvalid`; output is plain when redirected to a file or when the caller sets `NO_COLOR=1`. #162 (progress indicators) — add indicatif. Connect / query phases use indeterminate spinners; the write phase uses a bounded bar with row count and ETA. `logging::make_progress` returns `ProgressBar::hidden()` when `--quiet` is set, when stderr is not a TTY, or when `NO_COLOR` is set, so callers don't branch on whether progress is visible. Spinners finish-and-clear before tracing logs fire, keeping stderr greppable. #163 (tracing-subscriber) — add tracing + tracing-subscriber. Initialise a compact, plain-text stderr subscriber in `main()` before any work, with level filtering derived from `cli.verbose` / `cli.quiet`: `--quiet`→error, default→warn, `-v`→info, `-vv`→debug, `-vvv+`→trace. `RUST_LOG` overrides for ad-hoc debugging. Replace all four `eprintln!` sites in `src/main.rs` with `tracing::info!` (status) / `tracing::error!` (failures), and migrate the remaining `eprintln!` calls in `src/exit.rs` and `src/tls/{config,pool}.rs` to the same set of macros so `--quiet` actually suppresses everything except errors. Credential redaction is unchanged: `redact_sql_error` still scrubs `mysql::Error` strings before they reach `tracing::error!`. The subscriber does not inspect field contents — that's the caller's responsibility, documented in the new module header. cargo-deny: add `supports-color` (2.x ↔ 3.x divergence inside owo-colors 4.3) and `windows-sys` (0.52 ↔ 0.61 divergence across the new tracing / indicatif / clap subtrees) to `bans.skip` with explicit `reason = ...` strings tracking the upstream consolidation status. Snapshots: tls_cli_integration error snapshots now include the `ERROR` level prefix that tracing-subscriber prepends. Updated via `INSTA_UPDATE=always cargo test --test tls_cli_integration`. Test coverage: 5 new unit tests in `src/logging.rs` covering NO_COLOR gating (set / empty / unset), `--quiet` suppressing progress, NO_COLOR suppressing progress, and `init_tracing` idempotence. Existing 580-test suite continues to pass under `just ci-check`. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Splits `src/main.rs` (~750 lines mixing CLI dispatch, config resolution, TLS pool creation, query execution, output dispatch, completion generation, and test harness) into focused modules: - `src/main.rs` (~55 lines): parse CLI, init crypto + tracing, dispatch subcommands / `--dump-config`, hand off to `run::run(cli)`. - `src/run.rs`: query-execution pipeline (resolve → connect → query → write) + MySQL-error-code mapper. Calls `config`, `connection`, `output`. - `src/config.rs`: `resolve_database_url`, `resolve_database_query`, `resolve_output_file`, `dump_configuration`, `validate_query_file_path` and its size/extension constants. - `src/connection.rs`: `create_database_connection` (TLS-aware pool builder). - `src/completion.rs`: `generate_completion` for shell completions. - `src/output.rs`: `write_output`, `create_output_file`, open-error classifier. Unit tests moved alongside their subject (17 config, 3 connection, 4 completion = 24 tests). Each new module is published as `pub mod` under `src/lib.rs` so future test code can reach them directly. No public- binary behaviour changes: same CLI, same exit codes, same output formats, same error messages. Refs #14 (P1-K batch). Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Replace the eager `conn.query::<Row,_>` path (which materialised the full
result set into `Vec<mysql::Row>` before writing a single byte) with a
streaming pipeline that feeds `conn.query_iter` rows directly into a
format-specific `RowSink`. Peak memory is now linear in the column count,
not the row count — the previous 16 GB ceiling at ~2.3M JSON / ~3.5M CSV
rows no longer applies.
Pipeline shape
--------------
- New `src/sink.rs` defines the `RowSink` trait (`on_headers` / `on_row`
/ `finalize`) plus three implementations: `DelimitedSink` (CSV and
TSV share plumbing, differ only in delimiter) and `JsonSink`.
- Each sink writes to `<output>.tmp` and renames on
`finalize()`-success. `Drop` cleans up the `.tmp` on any failure,
preserving the atomic-output invariant — a type-conversion error on
row N now leaves **no** stale partial output behind.
- `src/run.rs::stream_query` opens `query_iter`, snapshots the column
metadata once, builds the sink lazily (post-parse, pre-rows) and
pushes rows into it. The progress indicator is now an indeterminate
spinner that updates its message every 1000 rows (we can't know the
total up-front with streaming).
- `src/output.rs` exposes `build_sink` for the live path and keeps
`write_output(Vec<Row>, …)` for snapshot tests that feed in-memory
rows. `write_output` now drives the same sink plumbing instead of
the old `rows_to_strings` → `csv::write` helpers.
Error semantics
---------------
- Type-conversion failures are wrapped as `GoldDiggerError::Query`
inside `sink::wrap_conversion_error` so exit code 4 stays stable
even if the upstream `TypeTransformer` message text is refactored.
- `mysql::Error` from row fetch (mid-stream network failure etc.)
flows through the same `map_query_error` + `redact_sql_error` path
as the initial query error, so credentials still cannot leak.
- `--allow-empty` semantics are preserved: `on_headers` fires up-front
so an empty stream can still be finalized to a valid envelope
(JSON `{"data":[]}`, CSV/TSV header-only).
Pretty-print & backward-compat
------------------------------
- `--pretty` JSON serialises each row with `to_writer_pretty` (per-row
indentation). The envelope `{"data":[` / `]}` markers stay compact;
a separate polish pass can add newlines between rows if the UX
warrants it.
- The public `rows_to_strings` function and the `write` free-fns in
`csv.rs` / `json.rs` / `tab.rs` are preserved — they are unchanged
and the existing snapshot tests still exercise them.
Benchmark note
--------------
- `benches/memory_ceiling.rs` is now a *historical* reference for the
old `rows_to_strings` path; the docstring is updated to say so. A
follow-up can either retire it or rewrite it to measure the streaming
pipeline end-to-end.
Tests
-----
- 8 new unit tests in `src/sink.rs` cover header/row writes, empty-
result envelopes for JSON, failure cleanup (`.tmp` removed, target
missing), the "refuse existing file without --force" guard, the
`--force` overwrite path, and exit-code routing for conversion
errors.
- `just ci-check` passes: 588 tests run, 588 passed (1 slow),
0 skipped; advisories/bans/licenses/sources ok.
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Removes the second-confirmation gate added by P1-D for --allow-invalid-certificate (todo #22). The companion flag (--i-understand-this-is-insecure) and GOLD_DIGGER_ALLOW_INVALID env gate are paternalistic — when a user explicitly passes a CLI flag, the binary should honour it. Stderr [DANGER] warnings remain (and obey --quiet / NO_COLOR / TTY detection); the 3-second pre-connect sleep is gone. Audit-time grep on deployment manifests is the right enforcement point, not binary-side structural overrides. Drops: - `i_understand_this_is_insecure` field from `cli::TlsOptions` - `GOLD_DIGGER_ALLOW_INVALID` env var - gating logic in `cli::TlsOptions::to_tls_config` (always proceeds with the user's choice) - `std::thread::sleep(3s)` in `tls::config::display_security_warnings` - doc trails in `connection.rs` referencing the gate - 28 `--i-understand-this-is-insecure` arg additions in `tests/integration/data_types.rs` - 13 `i_understand_this_is_insecure: ...` struct-init lines in `tests/tls_backward_compatibility.rs` and `tests/tls_config_unit_tests.rs` - the `test_cli_allow_invalid_certificate_requires_companion_flag` negative test (no longer applicable) Snapshot updates: cli_testing_example, tls_backward_compatibility, tls_cli_integration, and tls_config_unit_tests help/error snapshots regenerated to drop the removed flag from --help output. Also lands the P2-DOCS batch (14 todos): - AGENTS.md "Requirements Gap Analysis" reflects current state (F001-3, F005, F007, F008, F010 all closed); no longer references gitignored paths. - CHANGELOG [Unreleased] now lists CLI-first config, typed exit codes, streaming output, --output / --query-file safety guards, structured logging, coloured stderr, progress indicators, new CI workflows. - src/lib.rs: crate-level //! doc was already present; expanded init_crypto_provider rustdoc to document the silent-error-swallow trade-off; expanded get_required_env with a doctest example; documented case sensitivity gotcha on get_extension_from_filename. - src/exit.rs: map_error_to_exit_code rustdoc warns that error-message text is part of the public API in the substring-fallback path. - src/type_transformer.rs: added per-format usage table to TypeTransformer rustdoc. - src/tls/pool.rs: expanded create_tls_connection rustdoc with full Arguments / Returns / Errors / Example sections. - src/utils.rs: strengthened redact_sql_error doctest to assert URL userinfo replacement + idempotence. - src/cli.rs: TlsOptions field rustdocs now classify the residual attack surface for tls_ca_file (CA-key compromise) and insecure_skip_hostname_verify (CWE-297). - README.md: cross-link to tests/README.md and benches/README.md. just ci-check: 587/587 tests pass; advisories/bans/licenses/sources ok. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Resolves the P2 test cluster surfaced in the comprehensive code review. Each finding either gains targeted coverage or is consolidated against existing work landed in earlier P1 commits. New test surface: - tests/redact_sql_error_truth_table.rs: rstest one-row-per-pattern table for redact_sql_error, including idempotence and a non-ASCII (contraseña) sentinel. (#74) - tests/additional_mysql_types.rs: feature-gated coverage of DECIMAL precision, BIGINT u64::MAX/i64 boundaries, TIME signed/days aggregation, DATETIME microseconds, and NULL convergence. (#76) - tests/exit_code_1_no_rows.rs: Docker-gated E2E for empty-result exit contract -- default exits 1 with no output file committed, --allow-empty exits 0 with `{"data":[]}` JSON / header-only CSV. (#77) - tests/init_crypto_provider_idempotency.rs: serial + multi-thread idempotence pin for the `Once`-guarded rustls bootstrap. (#81) - tests/tls_danger_banner_snapshot.rs: insta snapshot of the `--allow-invalid-certificate` [DANGER] banner block, plus a positive assertion that --quiet does not suppress error-level lines. (#82) - tests/stream_separation.rs: stdout/stderr separation contract for --help, --version, --dump-config, completion, missing-config error, unreachable-host error, and the DANGER banner path. (#176) - tests/cli_testing_example.rs: insta-pinned exact clap mutex error for --verbose + --quiet. (#78) Bench: - benches/memory_ceiling.rs: heavy 1M/2M/3M-row sweep gated on `IGNORE_BENCHES=0`; documents the legacy buffered curve so streaming (F007) regressions remain visible. (#83) Snapshot path normalization (#79): - Move the 40 snapshots under `tests/tests/snapshots/` to `tests/snapshots/` and update with_settings! to use `snapshot_path => "snapshots"` so a future test-file move no longer recreates the doubled path. Already-resolved todos deleted: - #75 (cross-redactor fidelity table) -- subsumed by P1-C (commit 549f8e0) and existing tests/dump_config_redaction.rs. - #80 (migrate JSON snapshots from json::write to write_json_maps) -- subsumed by P1-E (commit bf7ac78). - #84 (.failure() -> .code(N) audit) -- subsumed by the P1-J sweep (no .failure() calls remain in tests/). - #55 (atomic CSV/TSV temp-file rename) -- subsumed by F007 streaming sink (commit f052c46) which uses `.tmp + rename` across all formats. Deferred: - #85 (delete tls_integration_old.rs): file is referenced by .github/workflows/ci.yml; deletion needs a paired CI edit which is out of scope for this batch. - #174 (consolidate TLS test corpus): 6-8 hour merge. - #175 (split data_types.rs 5151 LOC): 8-10 hour mechanical refactor. Verified via `just ci-check` (642 tests, 0 failures, 0 clippy warnings). Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Both .github/ci-performance-config.yml and .github/error-reporting-config.yml are not referenced by any workflow, issue template, or tool config -- leftover files from a discontinued tool. Removing them reduces noise in the repo root. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
CLI hardening (todos #154, #177, #152, #179): - Cli, Commands, Shell, OutputFormat now derive Debug for inspectability - Commands, Shell, OutputFormat marked #[non_exhaustive] for downstream semver future-proofing (new variants no longer require a major bump) - Shell and OutputFormat now derive Copy (cheap pass-by-value, removes needless .clone() calls -- output.rs updated to take advantage) - Cli root #[command(...)] now exposes author, long_about, after_help with EXAMPLES section -- richer --help output for users Documentation (todo #142): - TypeTransformer::value_to_string and value_to_json gain doctest-tested Example sections so docs.rs shows runnable samples Snapshot tests refreshed for the new --help output. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Release docs (#143, #156): - RELEASING.md: correct '6 platforms' -> '7 target triples' to match dist-workspace.toml (macOS arm64/x64, Linux arm64/x64-gnu/x64-musl, Windows arm64/x64). - RELEASING.md: add explicit `grep -rn 'v0\.[0-9]'` step to the pre-release checklist so version drift in AGENTS.md/CLAUDE.md/README.md is caught before tagging. Documentation cross-links (#146): - docs/src/development/api-reference.md: add canonical docs.rs link so external readers reach the published rustdoc when the local mdBook copy is unavailable. Markdown hygiene (#141): - AGENTS.md: replace four-backtick fence with a regular triple-backtick rust block + indented inner example. Renders correctly in older GitHub UIs and through mdformat. Snapshot workflow (#160): - justfile: add `insta-review` recipe (`INSTA_UPDATE=always cargo test`). - CONTRIBUTING.md: document the snapshot regen flow and call out that `cargo insta review --accept` is NOT the supported command here. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Replace `*.snap binary` with `*.snap text eol=lf -diff` so insta snapshot files stay byte-identical across macOS/Linux/Windows checkouts. The previous `binary` attribute hid line-ending churn but still let Windows checkouts produce CRLF-encoded snapshots that diverged from Linux/macOS golden output. Diffing remains intentionally suppressed -- contributors review snapshot changes via `just insta-review` (todo #160) rather than git diff. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Resolve P2 src-cleanup / refactor todos. Compiles together with the in-flight #68 (env snapshot), #169 (lazy crypto init), and #177 (non-exhaustive Commands wildcard) work because the underlying tree needed all three to compile cleanly after 77a448c. Resolved (mine): * #53/#60 — TlsError::from_rustls_error: switch user-facing messages to Display where rustls supports it; document Debug fallback for variants that don't (PeerMisbehaved, AlertDescription); add explicit fallthrough comment for the non_exhaustive rustls::Error. * #54 — new src/mysql_errors.rs with named ER_*/CR_* constants; classify_mysql_error_code helper extracts the error-code → message table out of map_query_error so it's grep-able and unit-tested. * #56 — document rows_to_strings empty-input behaviour and direct callers to crate::sink::json_sink for JSON envelope guarantees. * #58 — extract crate::delimited::write_delimited shared helper used by csv.rs and tab.rs; both writers collapse to one-line delegates. * #59 — pub const OUTPUT_BUFFER_CAPACITY in lib.rs; csv/tab/json/sink all reference the canonical knob. * #63 — rows_to_strings now delegates per-row conversion to TypeTransformer::row_to_strings so the column-level error message matches the streaming sink path. * #65 — Opts::from_url error path routes the URL through utils::redact_url and the error string through redact_sql_error so malformed-URL errors no longer leak credentials (CWE-532). * #66 — fail-closed regex validation: REDACTION_PATTERN_DEFS slice + EXPECTED_REDACTION_PATTERN_COUNT pin; debug builds panic on bad pattern, release builds tracing::error! and skip; new test asserts count. * #69 — TypeTransformer::row_to_json_with_columns helper takes a pre-extracted Arc<str> column-name slice; JsonSink populates the list once in on_headers and reuses it per row. * #70 — drop the outer BufWriter on csv.rs/tab.rs; the shared delimited helper sets WriterBuilder::buffer_capacity instead. * #71 — add itoa/ryu deps; TypeTransformer::value_to_string and format_special_float use stack buffers for integer/float formatting. * #72 — add cold-path note to map_error_to_exit_code so future readers don't micro-optimise the substring matcher. * #102 — closed (already optimised by 407bd0f's HEX_CHARS table). * #106 — migrate src/lib.rs INIT from std::sync::Once to OnceLock<()> for consistency with src/utils.rs. * #100 — already done by #5 streaming pipeline (f052c46). * #55 — already done: src/sink.rs uses <output>.tmp + atomic rename. * #61 — deferred: collapsing exit_with_error/exit_success/exit_no_rows into a single helper hurts call-site readability; current shape already routes through map_error_to_exit_code. * #107 — deferred: cosmetic cache resolved by typed classifier ([R-H3]). Resolved (parallel agent #20, P2-DOCS / config / crypto-init batch): * #68/S15 — EnvSnapshot reads DATABASE_* once at startup; downstream resolvers consume the snapshot. * #169 — install rustls crypto provider lazily inside create_database_connection; main.rs no longer pays the cost on --help / completion / --dump-config. * #177 — Commands #[non_exhaustive] wildcard arm + dump-config refactor through build_configuration_dump. Test plan * cargo check / cargo clippy --all-targets -- -D warnings: clean. * cargo test --lib: 178 passed. * cargo test (full): all integration + doctests pass. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Resolve four small P2/P3 todos surfaced by the comprehensive review: * #113 -- audit `.github/zizmor.yml` ignores: drop the stale `cargo-dist.yml` entry (workflow no longer exists, deleted in be42557) and add `# reason:` comments documenting why each remaining ignore exists. `codeql.yml` is preserved because the workflow file ships in this branch. * #121 -- extract `TLS_MODE_GROUP_ID` const for the clap mutex group shared by `--tls-ca-file`, `--insecure-skip-hostname-verify`, and `--allow-invalid-certificate`. Future renames are now a single edit. * #139 -- normalise test prefix in `tests/type_transformer_snapshots.rs` from `snapshot_*` to `test_*`. The 27 functions all use explicit `assert_snapshot!` literals, so snapshot file names are unchanged. `grep "^fn test_"` now finds the entire test surface in one sweep. * #147 -- README "Development Status" claimed "Advanced Testing ... planned" while ROADMAP marked v0.3.0 testing complete. README now matches ROADMAP and points readers to `tests/` and `benches/`. `just check` passes (633 tests, 0 clippy warnings, fmt clean). Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
The json and csv features were removed (Cargo.toml notes them as empty-marker flags that built byte-identical binaries) but several references still referenced them: - .github/workflows/ci.yml: 'test-features (minimal)' job and the coverage 'minimal features' step both passed --features 'json csv ...' which now errors with 'package does not contain these features'. - README.md, AGENTS.md: minimal-build example commands. Also fixes a parallel-test env-var race exposed by cargo llvm-cov in CI: test_resolve_database_query_from_env, test_resolve_database_query_missing, test_resolve_output_file_from_env, and test_resolve_output_file_missing called Cli::parse_from outside their temp_env::with_var closure. Clap's #[arg(env = ...)] reads the live env at parse time, so a sibling test mutating the same var could leak its value into cli.output / cli.query before the closure even started. Moving parse_from inside the closure puts both the read and the write under temp_env's serial mutex. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
There was a problem hiding this comment.
Warning
CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.
Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.
Actionable comments posted: 23
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
tests/integration/common.rs (1)
1-2015: 🧹 Nitpick | 🔵 TrivialFile exceeds recommended length guideline.
At 2015 lines, this file significantly exceeds the ≤1000 line (preferably ≤500) guideline. Consider splitting into separate modules:
cli.rsforGoldDiggerCliand command executionparsing.rsforOutputParserand format validationtemp.rsforTempFileManagerandTestIsolationpredicates.rsforCliPredicatesand validation predicatesThis would improve maintainability and reduce cognitive load when navigating the test utilities.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/integration/common.rs` around lines 1 - 2015, The file is too large (2015 lines); split it into focused modules to improve maintainability by extracting related types and functions: create cli.rs and move GoldDiggerCli (and its impls: execute, execute_with_assertions, execute_with_timeout, execute_raw, execute_with_output_validation, execute_with_snapshot, redact_sensitive_output) and AssertCmdIntegration into a cli module; create parsing.rs and move OutputParser, CsvParseResult/JsonParseResult/TsvParseResult, and OutputValidator + related predicate structs (CsvContentPredicates, JsonContentPredicates, TsvContentPredicates) into a parsing module; create temp.rs and move TempFileManager, TestIsolation, TestEnvironment, TempFileManager methods and TempFileManager Drop into a temp module; create predicates.rs and move CliPredicates and related predicate helpers into it; update the original tests/integration/common.rs to declare mod cli; mod parsing; mod temp; mod predicates; re-export the moved symbols (pub use crate::cli::GoldDiggerCli; etc.) and update use paths inside moved code to reference the new module boundaries (e.g., references to OutputParser in cli -> parsing::OutputParser or re-exported names), ensure ENV_VARS_TO_REMOVE and TestCase/OutputFormat/GoldDiggerResult remain accessible (keep them in common.rs or a small shared mod) and run cargo check to fix any import visibility issues.tests/integration/tls_tests.rs (2)
40-47:⚠️ Potential issue | 🟠 Major
std::process::exit(0)in test skip logic bypasses test framework.Using
std::process::exit(0)terminates the entire test process, not just the current test case. This can mask failures in other tests running in the same process and prevents proper test reporting.Consider using a proper test-skip mechanism or restructuring with conditional compilation:
🛠️ Proposed fix using early return with explicit skip
/// Skip test if Docker is not available -fn skip_if_no_docker() { +fn skip_if_no_docker() -> bool { if !is_docker_available() { println!("Skipping test: Docker not available"); - // Use proper test skipping mechanism - std::process::exit(0); // Exit gracefully for skipped tests + return true; } + false }Then update call sites to return early:
fn test_platform_certificate_store(#[case] db_flavor: &str) -> Result<()> { if skip_if_no_docker() { return Ok(()); } // ... rest of test }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/integration/tls_tests.rs` around lines 40 - 47, The helper skip_if_no_docker currently calls std::process::exit(0), which aborts the whole test run; change skip_if_no_docker to return a bool (e.g., fn skip_if_no_docker() -> bool) or Result to indicate skipping instead of exiting, remove the std::process::exit(0) call, and update callers (like test_platform_certificate_store) to check skip_if_no_docker() and return early (e.g., if skip_if_no_docker() { return Ok(()); }) so only the individual test is skipped and the test framework continues reporting other tests.
639-754:⚠️ Potential issue | 🔴 CriticalAdd missing imports for
CertificateValidatorandCertificateLoaderfrom the fixtures module.The
ephemeral_certificate_testsmodule usesCertificateValidatorandCertificateLoader(defined intests/fixtures/tls/mod.rs), but these types are not imported intests/integration/tls_tests.rs. Theuse super::*;in the submodule only brings in what's available in the parent scope, so this will fail to compile.Add to the imports in
tests/integration/tls_tests.rs:use crate::fixtures::tls::{CertificateValidator, CertificateLoader, EphemeralCertificate};Or import them separately if preferred. Without this, all test functions in
ephemeral_certificate_testswill fail with "cannot find type" errors.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/integration/tls_tests.rs` around lines 639 - 754, The tests in the ephemeral_certificate_tests module reference CertificateValidator, CertificateLoader, and EphemeralCertificate but do not import them, causing "cannot find type" compile errors; add imports for these symbols from the fixtures tls module (e.g., import CertificateValidator, CertificateLoader, EphemeralCertificate from crate::fixtures::tls or equivalent) at the top of tests/integration/tls_tests.rs so the submodule can resolve those types referenced in test_ephemeral_certificate_generation, test_certificate_loading_utilities, and test_certificate_validation_utilities.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/dependabot.yml:
- Around line 15-16: YAML lint fails due to spaces inside inline array brackets;
update the inline arrays (the keys patterns and update-types) to remove inner
spacing so they read like patterns: ["*"] and update-types: ["minor","patch"]
(also apply the same fix to the other occurrences mentioned for the file).
- Around line 30-39: The "security-sensitive" group aggregates multiple crates
into one PR, which contradicts the comment that each crate should be reviewed
and held back individually; update the Dependabot config so matching crates are
not batched by this group—either remove the "security-sensitive" group and
create distinct entries for each crate pattern (the listed "mysql*", "rustls*",
"clap*", "tokio*", "serde*") or change the group settings to disable aggregation
so each pattern produces its own PR (update the "security-sensitive" grouping
behavior referenced by the group name and the "patterns" list).
In @.github/workflows/benchmarks.yml:
- Around line 8-13: The YAML linter flags extra spaces inside branch list
brackets for the workflow triggers; update the bracketed branch arrays under the
on block (the pull_request and push entries) from "branches: [ main ]" to
"branches: [main]" to make formatting consistent and satisfy the linter while
leaving behavior unchanged.
In @.github/workflows/cargo-audit-pr.yml:
- Around line 10-12: The YAML lint fails due to bracket-spacing in the
workflow's branch lists; update the two occurrences of "branches: [ main ]" to
remove the inner spaces so they read "branches: [main]" or, alternatively,
convert them to explicit YAML sequences like "branches:\n - main"; update the
entries for both occurrences (the two "branches: [ main ]" lines) so the spacing
rule is satisfied.
In @.github/workflows/ci.yml:
- Around line 5-7: YAML lint fails due to spaces inside bracketed sequence
literals; update the branch and needs list syntaxes (the "branches:" entries and
any "needs:" list like the one around line 215) to remove inner spaces so
bracketed lists use the normalized form (e.g., change "[ main ]" and "[
test-tls, ... ]" to "[main]" and "[test-tls,...]" respectively) across the CI
workflow file to satisfy the bracket-spacing rule.
In @.github/workflows/codeql.yml:
- Around line 3-37: YAMLlint flags are caused by bracketed arrays with spaces
and the bare on key; update the workflow to use YAMLlint-safe styles by (1)
quoting the top-level key "on" (e.g., "on":) to avoid truthy-warning and (2)
normalize array syntax for keys like branches and language by removing spaces
inside brackets or converting to block lists (e.g., change branches: [ main ] to
branches: [main] or to branches: - main and matrix: language: - rust). Apply
these changes for all occurrences (branches, matrix.language, cron array) and
keep existing keys like concurrency, defaults, permissions, and jobs names
intact.
In @.github/workflows/coverage.yml:
- Around line 3-8: The bracket spacing in the workflow trigger arrays is
inconsistent; update the array literals under the on: push: branches and on:
pull_request: branches (and any similar arrays like workflow_dispatch if
present) to use compact bracket syntax (e.g., change "[ main ]" to "[main]") so
spacing matches the style used in benchmarks.yml.
In @.github/workflows/scorecard.yml:
- Line 15: The YAML lint error is caused by extra spaces inside the inline array
for the branches key; update the branches declaration (branches: [ "main" ]) to
remove the inner spaces so it becomes branches: ["main"] to satisfy YAMLlint's
inline-array spacing rule.
In @.github/workflows/security.yml:
- Around line 5-6: The YAML arrays "workflows: [ CI ]" and "types: [ completed
]" use inline spacing that fails YAMLlint; update them to the linter's
bracket-spacing style by normalizing spacing inside the brackets (e.g., remove
or adjust spaces around items so they match project lint rules) for the entries
currently shown as workflows and types in the security.yml workflow definition.
In `@AGENTS.md`:
- Around line 127-139: Replace the redundant phrase "comprehensive CLI
interface" in the Project Overview with a non-redundant alternative (e.g.,
"comprehensive CLI" or "comprehensive command-line interface"); locate the exact
text "comprehensive CLI interface" and update it so the description reads
cleanly (for example: "It features a comprehensive CLI" or "It features a
comprehensive command-line interface"), leaving surrounding lines like
"CLI-first with environment variable fallbacks using `clap`" unchanged.
In `@SECURITY.md`:
- Around line 101-104: The SECURITY.md claim that created output files are
always set to 0o600 conflicts with the repo policy to respect the system umask;
update the doc and/or implementation so they agree: either change the
SECURITY.md bullet ("0o600 permissions (Unix)") to state that files are created
with owner-only intent but the process respects the system umask (while still
enforcing O_CREAT | O_EXCL, O_NOFOLLOW and --force semantics), or change the
code that opens output files to apply the explicit 0o600 mode only after
clearing/applying the umask so the effective permissions respect the system
umask; ensure the wording references the same behaviors (O_CREAT | O_EXCL /
create_new, O_NOFOLLOW, --force, and the permission semantics) so documentation
and implementation match.
In `@src/completion.rs`:
- Around line 10-35: The public funcs generate_completion_to<W: Write> and
generate_completion are missing the required rustdoc sections; add standardized
/// docs for each including a short description plus # Arguments (documenting
shell and writer for generate_completion_to, shell for generate_completion), #
Returns (-> () ), and a # Example block (use no_run showing calling
generate_completion_to with a Vec<u8> writer and generate_completion with a
Shell variant), ensuring you reference the Shell type and Write bound in the
docs and keep the format consistent with other public APIs.
In `@src/connection.rs`:
- Around line 127-143: The tests test_create_database_connection and
test_create_database_connection_with_tls_options currently attempt real network
resolution against "nonexistent:3306", causing flaky DNS/timeouts; change them
to induce a deterministic parse/validation failure or move them to integration
tests. Specifically, update the test inputs used by create_database_connection
to a syntactically invalid DSN (e.g., a malformed URI) or otherwise trigger the
function's validation path so the call returns Err immediately instead of
attempting a network connect, or mark these as integration tests and remove them
from unit tests. Ensure the tests still assert result.is_err() and keep
references to TlsConfig and create_database_connection intact.
In `@src/delimited.rs`:
- Around line 54-65: The write and flush calls on the CSV writer lose context
because they use bare `?`; update the `wtr.write_record(row)?;` and
`wtr.flush()?;` calls to map errors into richer messages (e.g., include the
record index or contents and the operation) using `.map_err(...)` before the `?`
so failures from `wtr.write_record` and `wtr.flush` surface as contextualized
errors; target the `wtr` usage created via `WriterBuilder::new()` (variable
`wtr`) and ensure the returned error type matches the function's error
(wrap/convert as needed).
In `@src/logging.rs`:
- Around line 155-179: warn_banner and danger_banner double-allocate when colors
are supported because they call if_supports_color with a closure that returns a
String and then call .to_string() again; since you already check
stderr_supports_color(), remove the if_supports_color call and apply the style
directly to message (e.g., use message.yellow().to_string() for warn_banner and
message.red().bold().to_string() for danger_banner) and keep the else branch
returning message.to_owned(), referencing the warn_banner, danger_banner,
stderr_supports_color, and if_supports_color symbols to locate the code.
In `@src/tls/ca.rs`:
- Around line 123-161: The TOCTOU comes from checking ca_file_path.exists()
before File::open in load_ca_certificates; remove the exists() check and instead
attempt to open the file directly, mapping File::open errors into the
appropriate TlsError (e.g., map io::ErrorKind::NotFound to
TlsError::ca_file_not_found(ca_file_path.display().to_string()) and other I/O
errors to TlsError::invalid_ca_format with the error string) so behavior aligns
with CaFile::load and eliminates the race window.
In `@tests/additional_mysql_types.rs`:
- Around line 177-187: The test extended_types_null_paths_converge calls
TypeTransformer::value_to_string(&v).unwrap(); replace the .unwrap() with
.expect("value_to_string") to match the pattern used in other NULL tests (e.g.,
decimal_null_yields_empty_and_null), ensuring consistent failure messages;
update only the call in the test to use .expect with that string.
- Around line 116-154: Add a MySQL TIME maximum-boundary case to the cases array
in the time_boundaries_stable test: insert a tuple using Value::Time for the max
positive TIME (hours aggregated from days+h to 838:59:59) — e.g. (false, 34, 22,
59, 59, 0, "838:59:59.000000") — and optionally add the negative counterpart
(true, 34, 22, 59, 59, 0, "-838:59:59.000000"); then run the assertions that
call TypeTransformer::value_to_string and TypeTransformer::value_to_json to
ensure the transformer emits the canonical string for the extreme boundary.
In `@tests/credential_leak_sweep.rs`:
- Around line 46-53: LEAK_MARKERS currently lists common leak patterns but
misses "auth=" and "bearer="; update the LEAK_MARKERS constant to include
"auth=" and "bearer=" (e.g., add "auth=" and "bearer=" to the &[&str] array) so
tests detect those common credential patterns; ensure casing or trimming
behavior is consistent with existing entries in LEAK_MARKERS.
In `@tests/path_safety.rs`:
- Around line 150-202: The test function query_file_over_size_cap_rejected
declares a redundant variable size that is never passed into the assert!
message; either delete the unused size binding or actually use it in the
assertion by adding it to the format arguments and updating the format
placeholder (e.g., change "size_target={size}" to "{}" and append size as an
argument, or include size with a named format argument), ensuring the variable
name matches the placeholder or is removed entirely to avoid the unused-binding
warning.
In `@tests/stream_separation.rs`:
- Around line 122-137: The test connection_error_routes_to_stderr_only currently
only checks exit code and empty stdout but doesn't assert anything about stderr;
update the assertion chain on fresh_cmd() to also assert that stderr is
non-empty and contains the connection error diagnostic (e.g., use a predicate
like predicate::str::contains("connect") or "connection" or a suitable error
phrase) so the test fails if the diagnostic disappears; locate the assertion in
connection_error_routes_to_stderr_only and add the stderr(...) check after the
existing stdout(...) check.
In `@tests/test_support/cli.rs`:
- Around line 24-47: Add the required standardized doc sections to the two new
public helpers clean_cmd and clean_cmd_no_color: update each /// comment to
include an Arguments section (if none, state "None"), a Returns section
describing the returned assert_cmd::Command, and an Example section showing
minimal usage (example snippet invoking the function and running/asserting the
command); keep the existing descriptive text and ensure markdown formatting
(headings like "Arguments", "Returns", "Example") and triple-slash comments are
used for both functions.
- Around line 181-184: The test setup currently builds a Command inline and
manually removes ENV_VARS_TO_REMOVE; instead, refactor execute() to call the
existing clean_cmd() helper so command hygiene is centralized—locate the code
creating cmd in execute() and replace the manual env_remove loop with a call to
clean_cmd() (or have execute() delegate to clean_cmd() to obtain the sanitized
Command), ensuring you reference clean_cmd() and execute() so all env_scrub
logic lives in one place.
---
Outside diff comments:
In `@tests/integration/common.rs`:
- Around line 1-2015: The file is too large (2015 lines); split it into focused
modules to improve maintainability by extracting related types and functions:
create cli.rs and move GoldDiggerCli (and its impls: execute,
execute_with_assertions, execute_with_timeout, execute_raw,
execute_with_output_validation, execute_with_snapshot, redact_sensitive_output)
and AssertCmdIntegration into a cli module; create parsing.rs and move
OutputParser, CsvParseResult/JsonParseResult/TsvParseResult, and OutputValidator
+ related predicate structs (CsvContentPredicates, JsonContentPredicates,
TsvContentPredicates) into a parsing module; create temp.rs and move
TempFileManager, TestIsolation, TestEnvironment, TempFileManager methods and
TempFileManager Drop into a temp module; create predicates.rs and move
CliPredicates and related predicate helpers into it; update the original
tests/integration/common.rs to declare mod cli; mod parsing; mod temp; mod
predicates; re-export the moved symbols (pub use crate::cli::GoldDiggerCli;
etc.) and update use paths inside moved code to reference the new module
boundaries (e.g., references to OutputParser in cli -> parsing::OutputParser or
re-exported names), ensure ENV_VARS_TO_REMOVE and
TestCase/OutputFormat/GoldDiggerResult remain accessible (keep them in common.rs
or a small shared mod) and run cargo check to fix any import visibility issues.
In `@tests/integration/tls_tests.rs`:
- Around line 40-47: The helper skip_if_no_docker currently calls
std::process::exit(0), which aborts the whole test run; change skip_if_no_docker
to return a bool (e.g., fn skip_if_no_docker() -> bool) or Result to indicate
skipping instead of exiting, remove the std::process::exit(0) call, and update
callers (like test_platform_certificate_store) to check skip_if_no_docker() and
return early (e.g., if skip_if_no_docker() { return Ok(()); }) so only the
individual test is skipped and the test framework continues reporting other
tests.
- Around line 639-754: The tests in the ephemeral_certificate_tests module
reference CertificateValidator, CertificateLoader, and EphemeralCertificate but
do not import them, causing "cannot find type" compile errors; add imports for
these symbols from the fixtures tls module (e.g., import CertificateValidator,
CertificateLoader, EphemeralCertificate from crate::fixtures::tls or equivalent)
at the top of tests/integration/tls_tests.rs so the submodule can resolve those
types referenced in test_ephemeral_certificate_generation,
test_certificate_loading_utilities, and test_certificate_validation_utilities.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 89bb672f-fcbd-4d16-a2de-9da6b5746907
⛔ Files ignored due to path filters (61)
.cargo/audit.tomlis excluded by none and included by none.gitattributesis excluded by none and included by none.github/workflows/release.ymlis excluded by!.github/workflows/release.ymland included by.github/**Cargo.lockis excluded by!**/*.lockand included by nonebenches/README.mdis excluded by none and included by nonebenches/memory_ceiling.rsis excluded by none and included by nonebenches/memory_usage.rsis excluded by none and included by nonebenches/output_formats.rsis excluded by none and included by nonedocs/solutions/best-practices/drop-safe-atomic-file-output-rust-cli-2026-04-25.mdis excluded by none and included by nonedocs/solutions/best-practices/front-load-validation-with-resolved-config-type-2026-04-25.mdis excluded by none and included by nonedocs/solutions/best-practices/replace-substring-error-classifiers-with-typed-enums-2026-04-25.mdis excluded by none and included by nonedocs/solutions/best-practices/unified-credential-redaction-rust-binary-2026-04-25.mdis excluded by none and included by nonemise.lockis excluded by!**/*.lockand included by nonetests/snapshots/cli_testing_example__help_output.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/cli_testing_example__verbose_quiet_mutex_error.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/csv_empty_result_set.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/csv_escaping_quotes_and_commas.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/csv_newlines.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/csv_null_values.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/csv_standard_data.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/json_empty_result_set.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/json_large_integers.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/json_null_handling.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/json_pretty_printed.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/json_standard_data.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/tls_backward_compatibility__integration_compatibility_tests__cli_help_output.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/tls_cli_integration__tls_cli_flag_tests__invalid_ca_file_content_error.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/tls_cli_integration__tls_cli_flag_tests__nonexistent_ca_file_error.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/tls_cli_integration__tls_cli_flag_tests__tls_help_options.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/tls_danger_banner_snapshot__allow_invalid_certificate_danger_banner.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/tsv_null_conversion.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/tsv_special_characters.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/tsv_standard_data.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_json_bytes_valid_utf8.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_json_date.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_json_datetime_with_microseconds.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_json_double.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_json_float_infinity.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_json_float_nan.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_json_int.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_json_invalid_date_error.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_json_large_binary.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_json_null.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_json_time.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_json_uint_max.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_string_date.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_string_datetime.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_string_double.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_string_float.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_string_infinity.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_string_int.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_string_invalid_utf8.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_string_large_binary.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_string_nan.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_string_neg_infinity.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_string_null.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_string_time.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_string_time_with_days.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_string_uint.snapis excluded by!**/*.snapand included bytests/**tests/snapshots/value_to_string_utf8_bytes.snapis excluded by!**/*.snapand included bytests/**tests/tests/snapshots/json_null_handling.snapis excluded by!**/*.snapand included bytests/**
📒 Files selected for processing (83)
.cursor/rules/rust-best-practices.mdc.github/ci-performance-config.yml.github/dependabot.yml.github/error-reporting-config.yml.github/workflows/benchmarks.yml.github/workflows/cargo-audit-pr.yml.github/workflows/ci.yml.github/workflows/codeql.yml.github/workflows/copilot-setup-steps.yml.github/workflows/coverage.yml.github/workflows/scorecard.yml.github/workflows/security.yml.github/zizmor.ymlAGENTS.mdCHANGELOG.mdCONTRIBUTING.mdCargo.tomlREADME.mdRELEASING.mdSECURITY.mddeny.tomldist-workspace.tomldocs/src/development/api-reference.mddocs/src/development/architecture.mddocs/src/troubleshooting/type-errors.mddocs/src/usage/configuration.mdjustfilemise.tomlsrc/cli.rssrc/completion.rssrc/config.rssrc/connection.rssrc/csv.rssrc/delimited.rssrc/exit.rssrc/json.rssrc/lib.rssrc/logging.rssrc/main.rssrc/mysql_errors.rssrc/output.rssrc/run.rssrc/sink.rssrc/tab.rssrc/tls.rssrc/tls/ca.rssrc/tls/classifier.rssrc/tls/config.rssrc/tls/error.rssrc/tls/mod.rssrc/tls/pool.rssrc/type_transformer.rssrc/utils.rstessl.jsontests/additional_mysql_types.rstests/atomic_output.rstests/cli_testing_example.rstests/credential_leak_regression.rstests/credential_leak_sweep.rstests/dump_config_redaction.rstests/exit_code_1_no_rows.rstests/exit_code_3_auth.rstests/exit_code_4_query.rstests/exit_code_5_io.rstests/exit_codes.rstests/init_crypto_provider_idempotency.rstests/integration/common.rstests/integration/tls_tests.rstests/lib.rstests/output_format_snapshots.rstests/path_safety.rstests/redact_sql_error_truth_table.rstests/stream_separation.rstests/test_support/cli.rstests/tls_backward_compatibility.rstests/tls_classifier_rcgen.rstests/tls_cli_integration.rstests/tls_config_unit_tests.rstests/tls_danger_banner_snapshot.rstests/tls_integration_old.rstests/tls_variants_test.rstests/type_safety.rstests/type_transformer_snapshots.rs
💤 Files with no reviewable changes (3)
- .github/error-reporting-config.yml
- .github/ci-performance-config.yml
- src/tls.rs
GitHub's Default Setup CodeQL is enabled at the repo level (visible as the dynamic 'PR #N' workflow that already passes). The advanced .github/workflows/codeql.yml has been failing on every PR commit with: Code Scanning could not process the submitted SARIF file: CodeQL analyses from advanced configurations cannot be processed when the default setup is enabled The two cannot coexist. Default Setup provides equivalent (auto-pinned, auto-maintained) coverage for the same language matrix. Also drops the now-stale codeql.yml entry from .github/zizmor.yml's unpinned-uses ignore list. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
There was a problem hiding this comment.
Warning
CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.
Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/zizmor.yml:
- Around line 9-24: Update the .github/zizmor.yml suppressions so only
release.yml remains under unpinned-uses.ignore and remove docs.yml, ci.yml, and
security.yml from that list (they are SHA-pinned); also remove ci.yml from the
excessive-permissions suppression because its permissions are contents: read (no
write) and therefore should be evaluated normally; finally, adjust the
surrounding reason comment to state that only release.yml is intentionally
pinned by tag (v6.0.2) while the others are SHA-pinned and thus not suppressed.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 5fab9b9e-4901-4697-9dd3-bc43f8c62415
📒 Files selected for processing (1)
.github/zizmor.yml
| # reason: release.yml needs `contents: write` for tags/artifacts; | ||
| # docs.yml needs `pages: write` to publish mdBook; ci.yml's broad | ||
| # token surface is intentional for the matrix jobs. | ||
| - release.yml | ||
| - docs.yml | ||
| - ci.yml | ||
| unpinned-uses: | ||
| ignore: | ||
| # reason: third-party action versions are pinned by tag rather than | ||
| # SHA in these workflows; tracked separately in todo #013 (P1) for | ||
| # the release pipeline. The list intentionally drops the historical | ||
| # cargo-dist.yml entry — that workflow no longer exists (todo #113). | ||
| - release.yml | ||
| - cargo-dist.yml | ||
| - ci.yml | ||
| - codeql.yml | ||
| - docs.yml | ||
| - security.yml |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
workflows=(.github/workflows/release.yml .github/workflows/ci.yml .github/workflows/docs.yml .github/workflows/security.yml)
echo "== uses refs classification =="
for wf in "${workflows[@]}"; do
echo "-- $wf --"
rg -n '^\s*-\s*uses:\s*[^[:space:]]+@[^[:space:]]+' "$wf" | while IFS=: read -r ln txt; do
ref="$(sed -E 's/.*@([^[:space:]#]+).*/\1/' <<<"$txt")"
if [[ "$ref" =~ ^[0-9a-fA-F]{40}$ ]]; then kind="sha-pinned"; else kind="non-sha"; fi
echo "Line $ln [$kind] $ref"
done
done
echo
echo "== permissions blocks (root + jobs) =="
for wf in .github/workflows/ci.yml .github/workflows/docs.yml .github/workflows/release.yml; do
echo "-- $wf --"
rg -n -C2 '^\s*permissions:' "$wf"
doneRepository: EvilBit-Labs/gold_digger
Length of output: 1986
Remove SHA-pinned workflows from unpinned-uses suppressions and evaluate ci.yml in excessive-permissions.
ci.yml, docs.yml, and security.yml are all SHA-pinned, so they should not appear in unpinned-uses.ignore. Additionally, ci.yml has only contents: read permissions (no write scope), so its suppression under excessive-permissions should be removed.
Only release.yml should remain in unpinned-uses.ignore—it uses tag pins (v6.0.2) rather than SHAs. Update the reason comment to reflect this.
Proposed changes
rules:
excessive-permissions:
ignore:
# reason: release.yml needs `contents: write` for tags/artifacts;
- # docs.yml needs `pages: write` to publish mdBook; ci.yml's broad
- # token surface is intentional for the matrix jobs.
+ # docs.yml needs `pages: write` to publish mdBook.
- release.yml
- docs.yml
- - ci.yml
unpinned-uses:
ignore:
- # reason: third-party action versions are pinned by tag rather than
- # SHA in these workflows; tracked separately in todo `#013` (P1) for
- # the release pipeline. The list intentionally drops the historical
- # cargo-dist.yml entry — that workflow no longer exists (todo `#113`).
+ # reason: release.yml is still tag-pinned; migrate to full SHAs in
+ # todo `#013` (P1). cargo-dist.yml no longer exists (todo `#113`).
- release.yml
- - ci.yml
- - docs.yml
- - security.yml📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| # reason: release.yml needs `contents: write` for tags/artifacts; | |
| # docs.yml needs `pages: write` to publish mdBook; ci.yml's broad | |
| # token surface is intentional for the matrix jobs. | |
| - release.yml | |
| - docs.yml | |
| - ci.yml | |
| unpinned-uses: | |
| ignore: | |
| # reason: third-party action versions are pinned by tag rather than | |
| # SHA in these workflows; tracked separately in todo #013 (P1) for | |
| # the release pipeline. The list intentionally drops the historical | |
| # cargo-dist.yml entry — that workflow no longer exists (todo #113). | |
| - release.yml | |
| - cargo-dist.yml | |
| - ci.yml | |
| - codeql.yml | |
| - docs.yml | |
| - security.yml | |
| # reason: release.yml needs `contents: write` for tags/artifacts; | |
| # docs.yml needs `pages: write` to publish mdBook. | |
| - release.yml | |
| - docs.yml | |
| unpinned-uses: | |
| ignore: | |
| # reason: release.yml is still tag-pinned; migrate to full SHAs in | |
| # todo `#013` (P1). cargo-dist.yml no longer exists (todo `#113`). | |
| - release.yml |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/zizmor.yml around lines 9 - 24, Update the .github/zizmor.yml
suppressions so only release.yml remains under unpinned-uses.ignore and remove
docs.yml, ci.yml, and security.yml from that list (they are SHA-pinned); also
remove ci.yml from the excessive-permissions suppression because its permissions
are contents: read (no write) and therefore should be evaluated normally;
finally, adjust the surrounding reason comment to state that only release.yml is
intentionally pinned by tag (v6.0.2) while the others are SHA-pinned and thus
not suppressed.
YAML lint / workflow style: - Normalize bracket spacing across .github/dependabot.yml, benchmarks.yml, cargo-audit-pr.yml, ci.yml, coverage.yml, scorecard.yml, security.yml, ci.yml needs: list (no more `[ main ]` etc.). - Reflow split-line `run:` blocks in ci.yml minimal-features step, security.yml grype invocation, and coverage.yml llvm-cov to single lines so YAML plain-scalar folding is no longer load-bearing. - Remove security-sensitive grouping in dependabot.yml so mysql/rustls/ clap/tokio/serde updates ship as individual PRs (matches the comment's stated intent of per-crate review isolation). Code/correctness: - Cargo.toml: switch [profile.release] from panic=abort to panic=unwind. The atomic-output pipeline depends on Drop for .tmp cleanup (DelimitedSink/JsonSink/ProgressGuard); abort would silently leak partial files on panic. - src/completion.rs: add Arguments + Example rustdoc sections to the public generate_completion / generate_completion_to functions. - src/connection.rs: replace live-network 'nonexistent:3306' with 127.0.0.1:1 (tcpmux, guaranteed unbound) in two unit tests so they don't depend on DNS or network egress. - src/delimited.rs: wrap the per-row write and final flush in with_context() so failures report the row index instead of bare 'broken pipe'. - src/logging.rs: drop a double-allocation pattern in warn_banner / danger_banner -- inner closures now return borrowed Display wrappers so only the outer to_string allocates on the colored path. - src/tls/ca.rs: port the canonicalize-first / open-after TOCTOU hardening from CaFile::load into legacy load_ca_certificates so both APIs offer the same guarantee. Tightened parameter type to &Path. - AGENTS.md: drop redundant 'CLI interface' phrasing. Test hardening: - tests/test_support/cli.rs: GoldDiggerCommand::execute() now constructs via clean_cmd() instead of re-running cargo_bin_cmd! + env_remove, single-sourcing ENV_VARS_TO_REMOVE. Public helpers gained # Returns and # Example doc sections. - tests/integration/common.rs: fresh_cmd() returns Command directly (cargo_bin_cmd! is infallible), and delegates to test_support::cli::clean_cmd() instead of duplicating the env list. Added 'mod test_support;' to the five integration-test binaries that depend on it. - tests/exit_code_3_auth.rs, tests/exit_code_5_io.rs: replace hardcoded /tmp output paths with tempfile::tempdir() so tests are deterministic cross-platform and don't spuriously trip OutputExists (exit 2) when the expected failure mode is auth (3) or IO (5). - tests/additional_mysql_types.rs: pin both ends of MySQL TIME range (838:59:59 / -838:59:59); replace .unwrap() with .expect() for a named NULL-path assertion. - tests/credential_leak_sweep.rs: extend LEAK_MARKERS with auth=, bearer , and Bearer so the sweep covers URL query / form fields and HTTP Authorization headers. - tests/path_safety.rs: replace the unused size binding with named CAP_BYTES / EXPECTED_SIZE constants and an exact-size assert_eq so fixture drift is caught. - tests/stream_separation.rs: assert that connection failures actually appear on stderr (not just absent from stdout) so a future logging refactor can't silently misroute the breadcrumb. - tests/tls_classifier_rcgen.rs: drop unnecessary to_path_buf() calls after the load_ca_certificates signature tightened to &Path. Validation: just ci-check -- 752/752 tests passed (1 slow), deny-check clean. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
There was a problem hiding this comment.
Warning
CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.
Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.
Actionable comments posted: 6
♻️ Duplicate comments (1)
.github/workflows/security.yml (1)
5-5:⚠️ Potential issue | 🟡 MinorFix inline-array spacing on Line 5 to unblock YAMLlint.
workflows: [ CI ]still violates the bracket-spacing rule; useworkflows: [CI].Suggested fix
- workflows: [ CI ] + workflows: [CI]🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/security.yml at line 5, The inline array spacing in the YAML key "workflows" is violating YAMLlint's bracket-spacing rule; update the value used in the workflow declaration from "workflows: [ CI ]" to remove the spaces inside the brackets (i.e., "workflows: [CI]") so the inline array conforms to the linter rule.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/coverage.yml:
- Around line 10-12: The workflow currently only sets permissions: contents:
read but still relies on the CODECOV_TOKEN secret; update the permissions block
to include id-token: write (e.g., add "id-token: write" alongside contents:
read), change the Codecov action invocation to use OIDC by setting use_oidc:
true and remove the CODECOV_TOKEN secret input (or guard its removal by
switching to the OIDC flow in the codecov action call), and ensure any
references to CODECOV_TOKEN in the job steps are deleted so the action
authenticates via OIDC instead of the long‑lived secret.
In `@src/logging.rs`:
- Around line 56-84: Add the missing documentation sections for the new public
logging helpers: init_tracing, stderr_supports_color, warn_banner,
danger_banner, and make_progress. For each function add /// Arguments describing
parameters, /// Returns describing the return value (or unit), and an ///
Example showing minimal usage (copyable snippet) consistent with existing crate
style; keep wording concise and include any side-effects (e.g., global
subscriber installation in init_tracing or terminal output for banners). Ensure
the doc comments are placed immediately above each pub fn and follow the
project's three-section requirement (Arguments, Returns, Example).
In `@src/tls/ca.rs`:
- Around line 8-12: Imports in src/tls/ca.rs are misordered; reorder them to
follow the coding guideline: list std imports first (e.g., std::fs::File,
std::io::BufReader, std::path::{Path, PathBuf}), then external crate imports
(e.g., rustls_pki_types::{CertificateDer, pem::PemObject}), and finally local
module imports (e.g., use super::error::TlsError), with a blank line between
each group to match the project's import ordering convention.
In `@tests/credential_leak_sweep.rs`:
- Around line 163-166: Replace the hardcoded host-dependent missing CA path
"/nonexistent/path/to/ca.pem" used in the .args([...]) call in
tests/credential_leak_sweep.rs with a deterministic tempdir-relative path (e.g.,
create a TempDir via tempfile::tempdir() or use std::env::temp_dir() and join a
non-existent filename) and pass that joined path to the "--tls-ca-file" argument
so the test fails for a missing file consistently across platforms
(Windows/Linux/macOS).
In `@tests/exit_code_3_auth.rs`:
- Around line 28-51: The test constructs the Command by manually removing env
vars (cargo::cargo_bin_cmd!("gold_digger") with multiple .env_remove calls),
which duplicates scrub logic and can miss vars like RUST_LOG honored by
src/logging.rs::init_tracing; replace this manual setup with the shared helper
tests/test_support/cli.rs::clean_cmd() so the test calls
clean_cmd(cargo::cargo_bin_cmd!("gold_digger")) (or the equivalent pattern used
elsewhere) and remove the explicit .env_remove calls to ensure all scrubbed envs
(including RUST_LOG) are consistently applied and keep exit-code assertions
unchanged.
In `@tests/test_support/cli.rs`:
- Around line 16-23: The tests currently fail to scrub RUST_LOG from spawned
test processes; update the shared test helper so clean_cmd() also removes
"RUST_LOG" from the environment (i.e., add "RUST_LOG" to the ENV_VARS_TO_REMOVE
constant and ensure the code in clean_cmd() iterates that list and calls
.env_remove(...) for each entry) so parent-shell RUST_LOG won't leak into test
subprocesses or logging.rs behavior.
---
Duplicate comments:
In @.github/workflows/security.yml:
- Line 5: The inline array spacing in the YAML key "workflows" is violating
YAMLlint's bracket-spacing rule; update the value used in the workflow
declaration from "workflows: [ CI ]" to remove the spaces inside the brackets
(i.e., "workflows: [CI]") so the inline array conforms to the linter rule.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 6ff5f849-d075-4ba8-9015-660922455ea3
📒 Files selected for processing (28)
.github/dependabot.yml.github/workflows/benchmarks.yml.github/workflows/cargo-audit-pr.yml.github/workflows/ci.yml.github/workflows/coverage.yml.github/workflows/scorecard.yml.github/workflows/security.ymlAGENTS.mdCargo.tomlsrc/completion.rssrc/connection.rssrc/delimited.rssrc/logging.rssrc/tls/ca.rstests/additional_mysql_types.rstests/credential_leak_sweep.rstests/database_seeding_test.rstests/exit_code_3_auth.rstests/exit_code_5_io.rstests/integration/common.rstests/path_safety.rstests/stream_separation.rstests/test_execution_utilities.rstests/test_seed_data.rstests/test_support/cli.rstests/tls_classifier_rcgen.rstests/tls_integration.rstests/tls_variants_test.rs
| permissions: | ||
| contents: read | ||
|
|
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
For codecov/codecov-action v6, what are the exact workflow permission and input requirements for OIDC uploads (use_oidc: true), and are there any private-repo prerequisites?
💡 Result:
For codecov/codecov-action v6 with use_oidc: true: Workflow permissions required: permissions: id-token: write Input requirements: - uses: codecov/codecov-action@v6 with: use_oidc: true Any token provided is ignored; the action fetches the OIDC token using core.getIDToken('https://codecov.io') (audience defaults to Codecov's URL). Private repo prerequisites: None explicitly stated beyond the above. OIDC works equivalently to tokens for private repos (authenticates uploads). For PRs from forks, OIDC fails (no id-token access, similar to secrets), falling back to tokenless (may require Codecov settings for private repos, but OIDC itself has no extra private-repo setup like audience config in sources).
Citations:
- 1: https://github.com/codecov/codecov-action/tree/v6.0.0
- 2: https://github.com/codecov/codecov-action
- 3: https://github.com/marketplace/actions/codecov
- 4: how to enable openid connect with codecov and github actions? codecov/codecov-action#1489
- 5: codecov/codecov-action@d820d60
Migrate Codecov auth from long-lived token to OIDC.
The workflow currently grants only contents: read (lines 10-12) but depends on CODECOV_TOKEN secret (line 65). Adopt OIDC authentication instead: add id-token: write permission and use use_oidc: true in the codecov action. This eliminates the static CI secret, reducing credential exposure surface.
Proposed fix
permissions:
contents: read
+ id-token: write
@@
- name: Upload to Codecov
if: github.event_name != 'pull_request' ||
github.event.pull_request.head.repo.full_name == github.repository
uses: codecov/codecov-action@3f20e214133d0983f9a10f3d63b0faf9241a3daa # v6.0.0
with:
files: lcov.info
fail_ci_if_error: true
- token: ${{ secrets.CODECOV_TOKEN }}
+ use_oidc: true
slug: EvilBit-Labs/gold_digger🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/workflows/coverage.yml around lines 10 - 12, The workflow currently
only sets permissions: contents: read but still relies on the CODECOV_TOKEN
secret; update the permissions block to include id-token: write (e.g., add
"id-token: write" alongside contents: read), change the Codecov action
invocation to use OIDC by setting use_oidc: true and remove the CODECOV_TOKEN
secret input (or guard its removal by switching to the OIDC flow in the codecov
action call), and ensure any references to CODECOV_TOKEN in the job steps are
deleted so the action authenticates via OIDC instead of the long‑lived secret.
| /// Initialises the `tracing` subscriber with a plain-text stderr formatter | ||
| /// and a verbosity filter derived from CLI flags. | ||
| /// | ||
| /// # Arguments | ||
| /// | ||
| /// * `verbose` — the `-v` count from `cli.verbose` (u8). | ||
| /// * `quiet` — the `--quiet` flag from `cli.quiet`. | ||
| /// | ||
| /// # Level mapping | ||
| /// | ||
| /// | `quiet` | `verbose` | Level | | ||
| /// |---------|-----------|-------| | ||
| /// | `true` | *any* | error | | ||
| /// | `false` | 0 | warn | | ||
| /// | `false` | 1 | info | | ||
| /// | `false` | 2 | debug | | ||
| /// | `false` | ≥3 | trace | | ||
| /// | ||
| /// The `RUST_LOG` env var, when set, overrides the CLI-derived filter. | ||
| /// This is intended for ad-hoc debugging; CI / production should drive | ||
| /// verbosity through the CLI flags. | ||
| /// | ||
| /// # Idempotence | ||
| /// | ||
| /// Safe to call at most once per process. Subsequent calls after the first | ||
| /// successful installation silently no-op (the second `set_global_default` | ||
| /// fails and we discard the error rather than panic — the subscriber is a | ||
| /// best-effort observability concern, not a correctness-critical boundary). | ||
| pub fn init_tracing(verbose: u8, quiet: bool) { |
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major
Complete the required doc sections on the new public logging helpers.
init_tracing, stderr_supports_color, warn_banner, danger_banner, and make_progress still do not consistently provide the required Arguments, Returns, and Example sections. Please finish those while these APIs are still new.
As per coding guidelines, "src/**/*.rs: Doc comments (///) required for all public functions with Arguments, Returns, and Example sections`."
Also applies to: 129-206
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/logging.rs` around lines 56 - 84, Add the missing documentation sections
for the new public logging helpers: init_tracing, stderr_supports_color,
warn_banner, danger_banner, and make_progress. For each function add ///
Arguments describing parameters, /// Returns describing the return value (or
unit), and an /// Example showing minimal usage (copyable snippet) consistent
with existing crate style; keep wording concise and include any side-effects
(e.g., global subscriber installation in init_tracing or terminal output for
banners). Ensure the doc comments are placed immediately above each pub fn and
follow the project's three-section requirement (Arguments, Returns, Example).
| use super::error::TlsError; | ||
| use rustls_pki_types::{CertificateDer, pem::PemObject}; | ||
| use std::fs::File; | ||
| use std::io::BufReader; | ||
| use std::path::{Path, PathBuf}; |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Import ordering differs from coding guidelines.
The guidelines specify grouping imports by: std, external crates, then local modules (separated by newlines). Current order is local → external → std.
♻️ Suggested reordering
-use super::error::TlsError;
-use rustls_pki_types::{CertificateDer, pem::PemObject};
use std::fs::File;
use std::io::BufReader;
use std::path::{Path, PathBuf};
+
+use rustls_pki_types::{CertificateDer, pem::PemObject};
+
+use super::error::TlsError;As per coding guidelines: "Group imports by standard library, external crates, and local modules, separated by newlines."
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| use super::error::TlsError; | |
| use rustls_pki_types::{CertificateDer, pem::PemObject}; | |
| use std::fs::File; | |
| use std::io::BufReader; | |
| use std::path::{Path, PathBuf}; | |
| use std::fs::File; | |
| use std::io::BufReader; | |
| use std::path::{Path, PathBuf}; | |
| use rustls_pki_types::{CertificateDer, pem::PemObject}; | |
| use super::error::TlsError; |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/tls/ca.rs` around lines 8 - 12, Imports in src/tls/ca.rs are misordered;
reorder them to follow the coding guideline: list std imports first (e.g.,
std::fs::File, std::io::BufReader, std::path::{Path, PathBuf}), then external
crate imports (e.g., rustls_pki_types::{CertificateDer, pem::PemObject}), and
finally local module imports (e.g., use super::error::TlsError), with a blank
line between each group to match the project's import ordering convention.
| .args([ | ||
| "--tls-ca-file", | ||
| "/nonexistent/path/to/ca.pem", | ||
| "--db-url", |
There was a problem hiding this comment.
Use a tempdir-relative missing CA path here.
/nonexistent/path/to/ca.pem is host-dependent and awkward on Windows. A missing path under temp keeps the failure mode identical while making the test deterministic on every runner.
Suggested patch
let temp = tempdir().expect("tempdir");
let out = temp.path().join("out.json");
+ let missing_ca = temp.path().join("missing-ca.pem");
let url = sentinel_url();
let output = clean_cmd()
.args([
"--tls-ca-file",
- "/nonexistent/path/to/ca.pem",
+ missing_ca.to_str().expect("utf-8 path"),
"--db-url",
&url,As per coding guidelines, "tests/**: Review tests for: Cross-platform compatibility`."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tests/credential_leak_sweep.rs` around lines 163 - 166, Replace the hardcoded
host-dependent missing CA path "/nonexistent/path/to/ca.pem" used in the
.args([...]) call in tests/credential_leak_sweep.rs with a deterministic
tempdir-relative path (e.g., create a TempDir via tempfile::tempdir() or use
std::env::temp_dir() and join a non-existent filename) and pass that joined path
to the "--tls-ca-file" argument so the test fails for a missing file
consistently across platforms (Windows/Linux/macOS).
| let mut cmd = cargo::cargo_bin_cmd!("gold_digger"); | ||
| cmd.env_remove("DATABASE_URL") | ||
| .env_remove("DATABASE_QUERY") | ||
| .env_remove("OUTPUT_FILE") | ||
| .env_remove("NO_COLOR") | ||
| .args([ | ||
| "--db-url", | ||
| "mysql://baduser:badpass@127.0.0.1:1/db", | ||
| "--query", | ||
| "SELECT 1", | ||
| "--output", | ||
| // Point at a path inside a writable tempdir so the failure is | ||
| // unambiguously a DB-connection issue, not an output-write one. | ||
| output_path.to_str().expect("utf-8 path"), | ||
| ]) | ||
| .assert() | ||
| .code(3) | ||
| .stderr( | ||
| predicate::str::contains("connection") | ||
| .or(predicate::str::contains("connect")) | ||
| .or(predicate::str::contains("refused")) | ||
| .or(predicate::str::contains("MySQL")) | ||
| .or(predicate::str::contains("mysql")), | ||
| ); |
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major
Route these auth tests through clean_cmd() instead of re-listing env removals.
These blocks bypass the shared helper and hand-maintain the scrub list. That already risks drift with the current binary: src/logging.rs::init_tracing also honors RUST_LOG, so parent-shell logging can still bleed into these runs. Building both commands via tests/test_support/cli.rs::clean_cmd() keeps exit-code coverage deterministic and future-proof.
As per coding guidelines, "tests/**/*.rs: Use .env_remove(\"DATABASE_URL\") etc. on Command to prevent user shell env leakage in integration tests`."
Also applies to: 64-78
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tests/exit_code_3_auth.rs` around lines 28 - 51, The test constructs the
Command by manually removing env vars (cargo::cargo_bin_cmd!("gold_digger") with
multiple .env_remove calls), which duplicates scrub logic and can miss vars like
RUST_LOG honored by src/logging.rs::init_tracing; replace this manual setup with
the shared helper tests/test_support/cli.rs::clean_cmd() so the test calls
clean_cmd(cargo::cargo_bin_cmd!("gold_digger")) (or the equivalent pattern used
elsewhere) and remove the explicit .env_remove calls to ensure all scrubbed envs
(including RUST_LOG) are consistently applied and keep exit-code assertions
unchanged.
| /// Environment variables that Clap reads via `#[arg(env = "...")]`. These | ||
| /// must be removed from any spawned binary so that user-shell exports do not | ||
| /// leak into integration tests (recorded as a recurring failure mode in | ||
| /// project memory). `NO_COLOR` is included for forward compatibility with | ||
| /// upcoming color/ANSI work. | ||
| pub const ENV_VARS_TO_REMOVE: &[&str] = | ||
| &["DATABASE_URL", "DATABASE_QUERY", "OUTPUT_FILE", "NO_COLOR"]; | ||
|
|
There was a problem hiding this comment.
Scrub RUST_LOG in the shared test command helper.
clean_cmd() is now the single source of truth for test-process hygiene, but src/logging.rs also honors parent RUST_LOG. A developer shell with RUST_LOG=trace can therefore inject extra stderr into helper-based tests and destabilize snapshots/assertions even when the Clap env vars are cleared.
Suggested patch
-pub const ENV_VARS_TO_REMOVE: &[&str] =
- &["DATABASE_URL", "DATABASE_QUERY", "OUTPUT_FILE", "NO_COLOR"];
+pub const ENV_VARS_TO_REMOVE: &[&str] = &[
+ "DATABASE_URL",
+ "DATABASE_QUERY",
+ "OUTPUT_FILE",
+ "NO_COLOR",
+ "RUST_LOG",
+];As per coding guidelines, "tests/**/*.rs: Use .env_remove(\"DATABASE_URL\") etc. on Command to prevent user shell env leakage in integration tests`."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tests/test_support/cli.rs` around lines 16 - 23, The tests currently fail to
scrub RUST_LOG from spawned test processes; update the shared test helper so
clean_cmd() also removes "RUST_LOG" from the environment (i.e., add "RUST_LOG"
to the ENV_VARS_TO_REMOVE constant and ensure the code in clean_cmd() iterates
that list and calls .env_remove(...) for each entry) so parent-shell RUST_LOG
won't leak into test subprocesses or logging.rs behavior.
Summary
tech_debt_cleanupresolves ~145 review-backlog items spanning P1/P2/P3, plus 4 follow-up critical/high findings from a focused PR review of the result. 50 commits, 141 files changed (+13,022 / -4,291), 745/745 tests pass,cargo deny checkclean.The branch closes the four major architecture gaps previously flagged in
AGENTS.md:GoldDiggerErrorenum with downcast-based classifier; substring matching demoted to legacy fallback.RowSinktrait +conn.query_iterpipeline; output streams through<path>.tmpand atomic-renames on success. Peak memory is now O(1 row), not O(N rows).tracing+tracing-subscriber; every error string routed throughutils::redact_sql_errorbefore reaching the subscriber.ResolvedConfig::from_cliprojectsCliinto a fully-validated config exactly once at the binary boundary; downstream code never touchesCli.Plus 4 compound docs in
docs/solutions/best-practices/capturing the durable patterns from this work.Notable changes
Architecture
main.rsshrank from ~1050 LOC to 55 LOC; logic extracted intorun.rs,config.rs,connection.rs,completion.rs,output.rs,sink.rs,logging.rs.tls.rs(1911 LOC) split intosrc/tls/directory:mod,error,classifier,config,ca,pool.GoldDiggerError+ConfigError+TlsErrorenums replace ad-hocanyhow!strings at error origin sites.ResolvedConfig(withResolvedQuery,OutputTarget,EnvSnapshot) front-loads validation; mutually-exclusive flags become enum variants.RowSinktrait with per-format implementations (DelimitedSinkfor CSV/TSV,JsonSink); atomic<output>.tmp → renamecommit,Drop-based cleanup on error.CaFilenewtype validates PEM at construction;TlsConfigcollapsed intoTlsValidationMode.Security hardening
--query-filecanonicalised + size-capped (10 MiB default, configurable via--max-query-file-size) + extension deny-list;--outputusesO_NOFOLLOW+0o600+create_new(true);--forceopt-in to overwrite.RENAME_NOREPLACEon Linux,hard_link+unlinkon other Unix,OpenOptions::create_newon Windows; preserve.tmpon rename failure for user recovery.utils::redact_sql_error/redact_url;\bword-boundary patterns; cross-redactor placeholder consistency.mysql::Errorclassifier intls/pool.rs::classify_mysql_pool_errorreplaces ~80-line substring cascade.redact_urlfail-closed onset_password/set_usernamefailure (parseable-but-unredactable URLs returnREDACTED_URL_PLACEHOLDER).validate_timeusesu64arithmetic to preventu32overflow on hostile MySQLTimewire data.UX
tracing-subscriberstderr; coloured[DANGER]/[WARNING]banners (respectsNO_COLORand TTY detection);indicatifprogress spinner during connect / query / write phases.ProgressGuardRAII wrapper ensuresfinish_and_clearfires on every return path.Testing
assert_cmd).tests/credential_leak_regression.rs,credential_leak_sweep.rs,redact_sql_error_truth_table.rs,dump_config_redaction.rs— adversarial credential-leak coverage with high-entropy sentinels.tests/atomic_output.rs,path_safety.rs,sink_atomicity.rs— atomic-rename +O_NOFOLLOW+ state-guard coverage.tests/tls_classifier_rcgen.rs— rcgen-driven exhaustive classifier tests forrustls::Errorvariants.tests/init_crypto_provider_idempotency.rs,tls_danger_banner_snapshot.rs,additional_mysql_types.rs,stream_separation.rs— closed coverage gaps.src/exit.rs(1024 cases each) for typed-error stability.tests/test_support/cli.rs::clean_cmd()is the canonicalassert_cmd::Commandbuilder; deprecatedCommand::cargo_bin()callers migrated to thecargo::cargo_bin_cmd!macro.CI / devops
coverage.yml(80% threshold),cargo-audit-pr.yml(PR gate),benchmarks.yml(containerised MySQL),codeql.yml(Rust support).dependabot.yml: grouped github-actions updates, weekly cadence.cargo cyclonedxfor SBOM (replaces Syft to avoid stale-lockfile false positives).aws-lc-rs1.16.0 → 1.16.3 (cascadesaws-lc-sys0.37.1 → 0.40.0) to clear RUSTSEC-2026-0045/0046/0047/0048.mysqlandrustlsdeps; addednix(Linux-only),itoa,ryu,tracing,tracing-subscriber,indicatif,owo-colors,proptest(dev),rcgen(dev).Documentation
docs/solutions/best-practices/(new): 4 compound docs capturing the durable patterns:drop-safe-atomic-file-output-rust-cli-2026-04-25.md—process::exit+ Drop interaction, atomic-rename platform matrixreplace-substring-error-classifiers-with-typed-enums-2026-04-25.md— typed enum + downcast +anyhow::Error::from(typed).context(...)disciplinefront-load-validation-with-resolved-config-type-2026-04-25.md— parse-don't-validate Cli → ResolvedConfig at the binary boundaryunified-credential-redaction-rust-binary-2026-04-25.md— single redactor module, fail-closed regex compile, sentinel + corpus testingAGENTS.mdrefreshed: closed-gaps section reflects current state; staleresolve_config_valueblock replaced with description ofResolvedConfig::from_cli; "Documented Solutions" pointer added so future agents discoverdocs/solutions/.SECURITY.md: new "Operational Risk Surface" section documenting--allow-invalid-certificate(MITM exposure),--query-file(file-read scope),--dump-config(best-effort redaction).README.md: "Known Limits" section (memory profile pre-streaming),--allow-invalid-certificateMITM warning,--query-filesafety note, cross-links totests/README.mdandbenches/README.md.CHANGELOG.md[Unreleased]rewritten with breaking/behavior section + features section.Test plan
just ci-checkpasses locally: 745 tests, 0 failures, 0 skipped, advisories/bans/licenses/sources all okunsafeintroducedcargo fmt --checkcleaneprintln!remaining insrc/(all routed throughtracing)process::exitfrom insiderun::run/stream_query(soDropruns on error paths)mysql_errorinterpolation withoutredact_sql_error(verified bytests/credential_leak_sweep.rs)coverage.yml,cargo-audit-pr.yml,benchmarks.yml,codeql.ymlworkflows successfully on this PRNotes for review
This PR is intentionally large because the architecture changes are tightly coupled — the typed-error pipeline, the Result-returning
run, theDrop-safe sink, and theResolvedConfigboundary all reinforce each other. Splitting into smaller PRs would have produced an intermediate state where individual fixes were either unsafe (typed errors without preservation) or inert (atomic rename withoutDroprunning). Commits are individually buildable and atomic; reviewing commit-by-commit may be easier than diff-by-diff.Recommended commit-walk order for review:
1ea0c3d(foundation: typed errors + Result-returning run) — the substrate everything else builds one3b9e7e(sink atomicity) — TOCTOU + state guard + .tmp preservation5d6458e(type-transformer + redaction polish) — defensive hardening across multiple modulesb11dc7b(tls.rs split) — large but mechanical; tests prove behaviour preserved74e2a5d(main.rs extraction) — also large, also mechanicalf052c46(F007 streaming) — semantic change in the data pipelined42c7cb(ResolvedConfig) — the validation boundary620c729-6e53abb) capture the design rationale for items 1-7Open follow-ups intentionally NOT in this PR (tracked separately):
#013SHA-pinrelease.ymlactions (touches user's in-progress workflow file)#164SIGINT handler for clean shutdown (a SIGINT bypasses Drop, so.tmpfiles orphan on Ctrl-C; this is a separate signal-handling concern)