Skip to content

tech_debt_cleanup: resolve P1/P2/P3 review backlog (~145 todos) + 4 critical PR-review fixes#194

Open
unclesp1d3r wants to merge 57 commits into
mainfrom
tech_debt_cleanup
Open

tech_debt_cleanup: resolve P1/P2/P3 review backlog (~145 todos) + 4 critical PR-review fixes#194
unclesp1d3r wants to merge 57 commits into
mainfrom
tech_debt_cleanup

Conversation

@unclesp1d3r
Copy link
Copy Markdown
Member

Summary

tech_debt_cleanup resolves ~145 review-backlog items spanning P1/P2/P3, plus 4 follow-up critical/high findings from a focused PR review of the result. 50 commits, 141 files changed (+13,022 / -4,291), 745/745 tests pass, cargo deny check clean.

The branch closes the four major architecture gaps previously flagged in AGENTS.md:

  • F005 (exit codes): typed GoldDiggerError enum with downcast-based classifier; substring matching demoted to legacy fallback.
  • F007 (streaming output): RowSink trait + conn.query_iter pipeline; output streams through <path>.tmp and atomic-renames on success. Peak memory is now O(1 row), not O(N rows).
  • F008 (structured logging): tracing + tracing-subscriber; every error string routed through utils::redact_sql_error before reaching the subscriber.
  • F001-F003 (CLI-first config): ResolvedConfig::from_cli projects Cli into a fully-validated config exactly once at the binary boundary; downstream code never touches Cli.

Plus 4 compound docs in docs/solutions/best-practices/ capturing the durable patterns from this work.

Notable changes

Architecture

  • main.rs shrank from ~1050 LOC to 55 LOC; logic extracted into run.rs, config.rs, connection.rs, completion.rs, output.rs, sink.rs, logging.rs.
  • tls.rs (1911 LOC) split into src/tls/ directory: mod, error, classifier, config, ca, pool.
  • Typed GoldDiggerError + ConfigError + TlsError enums replace ad-hoc anyhow! strings at error origin sites.
  • ResolvedConfig (with ResolvedQuery, OutputTarget, EnvSnapshot) front-loads validation; mutually-exclusive flags become enum variants.
  • Streaming RowSink trait with per-format implementations (DelimitedSink for CSV/TSV, JsonSink); atomic <output>.tmp → rename commit, Drop-based cleanup on error.
  • CaFile newtype validates PEM at construction; TlsConfig collapsed into TlsValidationMode.

Security hardening

  • Path safety: --query-file canonicalised + size-capped (10 MiB default, configurable via --max-query-file-size) + extension deny-list; --output uses O_NOFOLLOW + 0o600 + create_new(true); --force opt-in to overwrite.
  • Atomic-rename TOCTOU fix: RENAME_NOREPLACE on Linux, hard_link+unlink on other Unix, OpenOptions::create_new on Windows; preserve .tmp on rename failure for user recovery.
  • Credential redaction unified through utils::redact_sql_error / redact_url; \b word-boundary patterns; cross-redactor placeholder consistency.
  • Typed mysql::Error classifier in tls/pool.rs::classify_mysql_pool_error replaces ~80-line substring cascade.
  • redact_url fail-closed on set_password/set_username failure (parseable-but-unredactable URLs return REDACTED_URL_PLACEHOLDER).
  • validate_time uses u64 arithmetic to prevent u32 overflow on hostile MySQL Time wire data.

UX

  • Structured tracing-subscriber stderr; coloured [DANGER] / [WARNING] banners (respects NO_COLOR and TTY detection); indicatif progress spinner during connect / query / write phases.
  • ProgressGuard RAII wrapper ensures finish_and_clear fires on every return path.

Testing

  • Test count: ~280 → 745 (+465 across the branch).
  • New end-to-end coverage for exit codes 1, 3, 4, 5 (each driven by real binary invocations against assert_cmd).
  • tests/credential_leak_regression.rs, credential_leak_sweep.rs, redact_sql_error_truth_table.rs, dump_config_redaction.rs — adversarial credential-leak coverage with high-entropy sentinels.
  • tests/atomic_output.rs, path_safety.rs, sink_atomicity.rs — atomic-rename + O_NOFOLLOW + state-guard coverage.
  • tests/tls_classifier_rcgen.rs — rcgen-driven exhaustive classifier tests for rustls::Error variants.
  • tests/init_crypto_provider_idempotency.rs, tls_danger_banner_snapshot.rs, additional_mysql_types.rs, stream_separation.rs — closed coverage gaps.
  • proptest invariants in src/exit.rs (1024 cases each) for typed-error stability.
  • Helper consolidation: tests/test_support/cli.rs::clean_cmd() is the canonical assert_cmd::Command builder; deprecated Command::cargo_bin() callers migrated to the cargo::cargo_bin_cmd! macro.

CI / devops

  • New workflows: coverage.yml (80% threshold), cargo-audit-pr.yml (PR gate), benchmarks.yml (containerised MySQL), codeql.yml (Rust support).
  • dependabot.yml: grouped github-actions updates, weekly cadence.
  • cargo cyclonedx for SBOM (replaces Syft to avoid stale-lockfile false positives).
  • Bumped aws-lc-rs 1.16.0 → 1.16.3 (cascades aws-lc-sys 0.37.1 → 0.40.0) to clear RUSTSEC-2026-0045/0046/0047/0048.
  • Bumped mysql and rustls deps; added nix (Linux-only), itoa, ryu, tracing, tracing-subscriber, indicatif, owo-colors, proptest (dev), rcgen (dev).

Documentation

  • docs/solutions/best-practices/ (new): 4 compound docs capturing the durable patterns:
    • drop-safe-atomic-file-output-rust-cli-2026-04-25.mdprocess::exit + Drop interaction, atomic-rename platform matrix
    • replace-substring-error-classifiers-with-typed-enums-2026-04-25.md — typed enum + downcast + anyhow::Error::from(typed).context(...) discipline
    • front-load-validation-with-resolved-config-type-2026-04-25.md — parse-don't-validate Cli → ResolvedConfig at the binary boundary
    • unified-credential-redaction-rust-binary-2026-04-25.md — single redactor module, fail-closed regex compile, sentinel + corpus testing
  • AGENTS.md refreshed: closed-gaps section reflects current state; stale resolve_config_value block replaced with description of ResolvedConfig::from_cli; "Documented Solutions" pointer added so future agents discover docs/solutions/.
  • SECURITY.md: new "Operational Risk Surface" section documenting --allow-invalid-certificate (MITM exposure), --query-file (file-read scope), --dump-config (best-effort redaction).
  • README.md: "Known Limits" section (memory profile pre-streaming), --allow-invalid-certificate MITM warning, --query-file safety note, cross-links to tests/README.md and benches/README.md.
  • CHANGELOG.md [Unreleased] rewritten with breaking/behavior section + features section.

Test plan

  • just ci-check passes locally: 745 tests, 0 failures, 0 skipped, advisories/bans/licenses/sources all ok
  • No unsafe introduced
  • No new clippy warnings
  • cargo fmt --check clean
  • Snapshot tests up to date (insta)
  • No eprintln! remaining in src/ (all routed through tracing)
  • No process::exit from inside run::run / stream_query (so Drop runs on error paths)
  • No raw mysql_error interpolation without redact_sql_error (verified by tests/credential_leak_sweep.rs)
  • CI runs the new coverage.yml, cargo-audit-pr.yml, benchmarks.yml, codeql.yml workflows successfully on this PR
  • CodeRabbit review pass (per project convention)
  • Manual smoke test against a real MySQL instance pre-merge

Notes for review

This PR is intentionally large because the architecture changes are tightly coupled — the typed-error pipeline, the Result-returning run, the Drop-safe sink, and the ResolvedConfig boundary all reinforce each other. Splitting into smaller PRs would have produced an intermediate state where individual fixes were either unsafe (typed errors without preservation) or inert (atomic rename without Drop running). Commits are individually buildable and atomic; reviewing commit-by-commit may be easier than diff-by-diff.

Recommended commit-walk order for review:

  1. 1ea0c3d (foundation: typed errors + Result-returning run) — the substrate everything else builds on
  2. e3b9e7e (sink atomicity) — TOCTOU + state guard + .tmp preservation
  3. 5d6458e (type-transformer + redaction polish) — defensive hardening across multiple modules
  4. b11dc7b (tls.rs split) — large but mechanical; tests prove behaviour preserved
  5. 74e2a5d (main.rs extraction) — also large, also mechanical
  6. f052c46 (F007 streaming) — semantic change in the data pipeline
  7. d42c7cb (ResolvedConfig) — the validation boundary
  8. The compound docs (620c729-6e53abb) capture the design rationale for items 1-7

Open follow-ups intentionally NOT in this PR (tracked separately):

  • #013 SHA-pin release.yml actions (touches user's in-progress workflow file)
  • #164 SIGINT handler for clean shutdown (a SIGINT bypasses Drop, so .tmp files orphan on Ctrl-C; this is a separate signal-handling concern)
  • A handful of P3 polish items deferred for v0.4.0+

unclesp1d3r and others added 30 commits April 22, 2026 22:27
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
…ling

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
- #26 (P4/R-M5): Add lto="fat" and panic="abort" to [profile.release];
  drop now-redundant lto="thin" from [profile.dist]. No should_panic tests.
- #27 (P5): Replace per-byte format!("{:02x}") in type_transformer with
  static HEX_CHARS lookup + preallocated String. Output bytes unchanged.
- #40 (D-H4/D-H5): Remove obsolete "exit(-1)=255" claim and rewrite the
  main.rs entry point section to describe CLI-first + env fallback +
  EXIT_CONFIG_ERROR=2 contract.
- deps: aws-lc-rs 1.16.0 -> 1.16.3 (cascades aws-lc-sys 0.37.1 -> 0.40.0),
  clearing RUSTSEC-2026-0045/0046/0047/0048 (PKCS7 verify / CRL scope
  bypasses). Minor bump, no API changes. Required to make cargo deny
  check (part of just ci-check) pass.

just ci-check: advisories ok, bans ok, licenses ok, sources ok;
479/479 tests pass.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
- #37 (D-H1): Add //! module-level docstrings to all 10 src files
  (lib, main, cli, exit, tls, type_transformer, json, csv, tab, utils).
  Each preamble summarises the module's purpose, public contract, and key
  invariants. Replaces a misplaced `///` doc comment at the top of
  utils.rs that was silently attached to a `use` statement.

- deny.toml: Remove windows-sys multi-version skip (only 0.61.2 remains
  after recent dep updates — skip was flagging as unnecessary-skip).
  Clarify webpki 0.22.4 as ISC via [[licenses.clarify]] — webpki ships
  a LICENSE file but no `license` field in Cargo.toml, producing a
  no-license-field warning. LICENSE content verified (hash 0x001c7e6c).

just ci-check: advisories ok, bans ok, licenses ok, sources ok;
479/479 tests pass, zero warnings.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Co-authored-by: Copilot <copilot@github.com>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
- #9 (D-C1): Remove stale "Pattern Matching Bug" claim from AGENTS.md
  (Other Critical Issues section + Quick Reference table). Bug was fixed
  long ago; src/main.rs uses correct Some(_) pattern.
- #10 (D-C2): Remove duplicate TLS-migration narrative from CHANGELOG
  [Unreleased] (the same content already lives in v0.2.6 - 2025-09-04).
  Drop the contradictory "CHANGELOG vs Cargo.toml" version-mismatch
  claim from AGENTS.md and the hardcoded "v0.2.6" architecture heading.
- #36 (T-H9): Add three regression tests in src/main.rs::tests for the
  CLI-overrides-env precedence path (db_url, query, output_file). Pins
  the historical Some(&_) pattern bug.
- #38 (D-H2): Document FormatWriter trait's actual scope in lib.rs --
  only JsonWriter implements it; csv/tab use the free-function pattern.
  Resolution of the trait-vs-module question deferred to todo #20.
- #39 (D-H3): Fix the format-module writer contract in AGENTS.md to
  match the actual signature (Item = String, output: W by value, not
  impl AsRef<str> / &mut W). Cursor rules file updated locally but is
  gitignored.
- #41 (D-H6): Document --allow-invalid-certificate MITM exposure in
  SECURITY.md (new "Operational Risk Surface" section), README.md (TLS
  example + note under CLI options), and src/cli.rs long-help.
- #42 (D-H7): Document --query-file path-read scope in the same
  SECURITY.md section, README.md CLI options table, and src/cli.rs
  long-help. Pairs with todo #23 (sandboxing).
- #43 (D-H8): Add "Known Limits" section to README.md and "Memory
  Profile (F007)" to docs/src/usage/configuration.md with concrete
  rows-vs-RAM table and mitigation guidance until F007 streaming lands.
- #44 (D-H9): Document --dump-config best-effort redaction caveat in
  SECURITY.md, configuration.md, src/cli.rs long-help, and the README
  example comment. Pairs with todos #004 and #29.

Snapshot updates: cli_testing_example__help_output,
tls_backward_compatibility__cli_help_output, and
tls_cli_integration__tls_help_options now reflect the new --help prose
for --query-file, --dump-config, and --allow-invalid-certificate.

just ci-check: 482/482 tests pass, advisories/bans/licenses/sources all
ok, zero warnings.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Adds a `GoldDiggerError` enum in `src/exit.rs` with stable
`exit_code() -> i32` mapping for the public 0-5 contract:
NoRows -> 1, Config -> 2, DbAuth -> 3, Query -> 4, Io -> 5,
Tls -> 3 (or 2 for CaFileNotFound / InvalidCaFormat /
MutuallyExclusiveFlags).

`map_error_to_exit_code` is reworked to two-pass classification:

  1. Typed path (preferred): walks the anyhow error chain and
     downcasts to `GoldDiggerError`, `TlsError`, or `std::io::Error`.
     Returns the variant's exit code. Stable under message-text
     refactors.
  2. Substring fallback (legacy): the existing keyword classifier
     runs only when no typed variant is found in the chain. Slated
     for removal once all construction sites are migrated (tracked
     under todos #17 - tls.rs 80-line classifier, #31 - test
     sweep, #165 - integration coverage for codes 3 and 5).

Migrates the three resolve_* functions in main.rs to construct
`GoldDiggerError::Config` and `GoldDiggerError::Io` directly so the
typed path is exercised in production from the first error site
users hit. Existing message text is preserved (the new wrapper is
"configuration error: <original>"), so the resolver tests'
substring assertions still pass.

Adds five new tests in `src/exit::tests` that exercise the typed
path independent of message text:
- test_typed_error_exit_codes_direct
- test_typed_error_through_anyhow_chain
- test_typed_path_overrides_substring_match
- test_io_error_downcast_path
- test_no_rows_variant_overrides_default_query_classification

Existing six substring tests retained as they exercise the legacy
fallback path that still serves un-migrated call sites.

just ci-check: 487/487 tests pass, advisories/bans/licenses/sources
all ok, zero warnings.

Closes #2 (foundation). Substring fallback removal blocked on
typed migration of remaining sites (#17, #31, #165).

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
…rage

Resolves P1-J batch (test hygiene + proptest, 5 todos):

- #7/#32: add proptest dev-dep (1.11) and five property tests in
  src/exit.rs guaranteeing (a) the substring classifier returns a code
  in [1..=5] for any input and never panics, and (b) the typed
  GoldDiggerError::{Config,DbAuth,Query} variants produce a stable exit
  code regardless of message text, even through anyhow::Error context
  layers. 1024 cases per test, ~0.26s total.

- #31: sweep five bare .failure() calls in tests/cli_testing_example.rs
  (no remaining .failure() anywhere under tests/). Each site now asserts
  the specific expected exit code per src/exit.rs contract: .code(2) for
  missing-config / Clap mutex errors, .code(3) for unreachable-host
  connection failures. Refactor the cli_command rstest fixture to
  env_remove DATABASE_URL / DATABASE_QUERY / OUTPUT_FILE / NO_COLOR.

- #31: add clean_cmd() helper to tests/test_support/cli.rs and
  env_remove inside GoldDiggerCommand::execute() so every command built
  through the support module starts from a known-clean environment.

- #165: new tests/exit_code_3_auth.rs asserts unreachable-host runs exit
  with EXIT_DB_AUTH_ERROR (3). New tests/exit_code_5_io.rs asserts
  chmod-0o000 query-file dir produces EXIT_IO_ERROR (5) on Unix, with a
  Windows fallback that points --query-file at a path whose parent is a
  regular file. Both use cargo::cargo_bin_cmd!(); Unix path skips when
  running as root.

- #166: audited all 10 test files that spawn the binary. Three files
  (tls_cli_integration.rs, tls_backward_compatibility.rs,
  tests/integration/common.rs) were missing env_remove — each now has a
  local fresh_cmd() helper that strips the four Clap-bound env vars. The
  cli_testing_example fixture was patched likewise. The remaining six
  files (exit_codes.rs, exit_code_3_auth.rs, exit_code_5_io.rs,
  credential_leak_regression.rs, dump_config_redaction.rs,
  test_support/cli.rs) already env_removed or do so via clean_cmd.

Also migrates 14 deprecated Command::cargo_bin("gold_digger") call sites
to the cargo_bin_cmd! macro form (or to fresh_cmd / clean_cmd helpers
that wrap it), matching commit f3021b3's typed-error preference.

Verification: `just ci-check` -> 546 tests pass (17 in src::exit::tests
including the five new proptests), advisories/bans/licenses/sources ok,
TLS validation passes.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
…uites (P1-C)

Consolidates the three divergent credential-redaction implementations
(utils::redact_sql_error, tls::redact_url, main::dump_configuration
inline) into a single module (src/utils.rs) so pattern fixes land
everywhere at once. The previously-weakest redactor lived in the
highest-risk output surface (--dump-config); it now routes through
the same regex set as the mysql error path.

Changes:

- src/utils.rs: adds redact_url (moved from tls.rs), redact_dump_query,
  and shared placeholder constants. Extends pattern set with pwd=,
  passwd=, pass=, SET PASSWORD, IDENTIFIED WITH ... BY, Kennwort,
  mot_de_passe, and contraseña. Adds idempotence and non-English tests.
- src/tls.rs: Pool::new map_err now funnels every mysql_error string
  through utils::redact_sql_error before interpolation. tls::redact_url
  becomes a thin re-export to utils::redact_url. Closes CWE-532/209
  leak in 80-line auth/URL/handshake wrap (todo #3, S1).
- src/main.rs: dump_configuration replaces the substring heuristic
  ('password' / 'identified by') that either missed GRANT/SET
  PASSWORD/pwd/passwd/base64/JWT/non-English OR obliterated the
  legitimate query with a sentinel. Now delegates to redact_dump_query
  so redaction is surgical, not nuclear (todos #004, #18).
- tests/credential_leak_regression.rs: sentinel-based E2E suite
  asserting no password/username sentinel reaches stdout/stderr across
  6+ failure surfaces (unreachable host, bad port, URL-encoded pw,
  malformed scheme, query-file with credentials, --dump-config with
  sensitive DATABASE_URL/DATABASE_QUERY). Synthesizes the server-side
  auth-denied string to pin the redactor contract without a live DB
  (todos #6, #35).
- tests/dump_config_redaction.rs: rstest truth table of 15 adversarial
  query shapes driven through --query and DATABASE_QUERY paths;
  asserts identical redaction fidelity between the two, plus a
  pass-through test for benign queries (todo #29).

Resolves todos #3, #004, #6, #16, #18, #29, #35.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
…lows

Resolves the CI/devops slice of the P1 review backlog (todos #34, #48,
#49, #50, #51, #52):

- tests: drop `if is_ci() { return Ok(()) }` skip guards from container
  tests in `type_safety.rs`, `tls_integration_old.rs`,
  `integration/tls_tests.rs`, and `tls_variants_test.rs`. Real TLS and
  type-safety flows now execute on Linux GitHub Actions runners (Docker
  is available) instead of passing silently. Per-platform skip is
  delegated to the runner matrix and `is_docker_available()`.
- coverage workflow: new `.github/workflows/coverage.yml` invokes
  `cargo llvm-cov --fail-under-lines 80` so coverage regressions block
  PR merges instead of degrading silently behind Codecov uploads.
- cargo-audit PR gate: new `.github/workflows/cargo-audit-pr.yml` runs
  the `actions-rust-lang/audit` action on `pull_request` and `push` to
  `main`, closing the window between merge and the next daily
  `audit.yml` cron tick.
- benchmarks workflow: new `.github/workflows/benchmarks.yml` compiles
  Criterion benches on PR (`--no-run`) and runs the full bench suite on
  `main` against a containerized `mysql:8.4` service, uploading results
  as artifacts so DB-backed benches (`rows_processing`,
  `value_conversion`) finally execute in CI.
- CodeQL: new `.github/workflows/codeql.yml` enables the GitHub Rust
  CodeQL ruleset on PR, push to `main`, and weekly schedule. The file
  was already on the zizmor unpinned-uses ignore list.
- dependabot: switch the github-actions ecosystem from daily/limit-10
  to weekly/limit-5 with a `groups:` block that batches minor/patch
  bumps, and add a Cargo grouping that isolates security-sensitive
  crates (mysql/rustls/clap/tokio/serde patches) from the bulk
  `rust-deps` group. Cuts the recent 4-of-5 individual-bump commit
  noise without blurring security-sensitive review.

Skipped from this batch:
- todo #13 (SHA-pin release.yml + zizmor exemption removal) requires
  editing `.github/workflows/release.yml`, which is currently part of
  the maintainer's in-progress workflow edits. Left in place.

All new workflows pin third-party actions to the full SHAs already
established in `dist-workspace.toml` and the existing workflows. Local
`just ci-check` passes (546/546 tests, deny + TLS validation green).

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
…P1-G)

#25 (rows_to_strings drain): switch the conversion loop from
`rows.iter()` to `rows.drain(..)` so each `mysql::Row` is dropped as
soon as its string representation has been extracted. Peak memory
during conversion now scales with the result vector alone (~1x raw
payload) rather than holding the source `Vec<Row>` and the fully
materialised `Vec<Vec<String>>` simultaneously (~2x). For large
result sets the saving is roughly half of the source-row footprint;
on the synthetic 100k row x 10 col case the resident-set delta drops
into the streaming-friendly band recorded by the new benchmark.
The function signature is unchanged (still `Vec<Row>` by value); the
single internal binding is now `mut rows`.

#33 (memory_ceiling benchmark): add `benches/memory_ceiling.rs`
exercising `rows_to_strings` at 1k / 10k / 100k synthetic rows. The
benchmark builds rows in-process via `mysql_common::row::new_row`
(no live MySQL required) and reports RSS via `sysinfo` plus an
estimated `Vec<Vec<String>>` footprint, so future F007 streaming work
has a regression-guard baseline. Registered in `Cargo.toml` under
`[[bench]] name = "memory_ceiling" harness = false`. Numbers, not
assertions -- the benchmark is a guard, not a gate.

Acceptance criteria addressed
- #25: rows dropped progressively during conversion; output stable;
  `just ci-check` green (546 tests passing).
- #33: bench compiles, runs without OOM at 100k rows, prints stable
  size/RSS lines for trend tracking; covered by `cargo build --benches`.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
…second-confirmation gate

Resolves 7 P1 todos around TLS error classification, security hardening, and
layer coupling:

- #17: Replace the 80-line `to_lowercase().contains(...)` closure in
  `Pool::new` with a typed match on `mysql::Error` variants (`IoError`,
  `TlsError`, `MySqlError`, `DriverError`, `UrlError`, `CodecError`, etc.).
  TLS wrapper errors are unwrapped and delegated to
  `TlsError::from_rustls_error`; all user-facing strings still flow through
  `utils::redact_sql_error` (P1-C redaction closure preserved).
- #12: Remove all `format!("{:?}", ...)` Debug-based classification from
  `from_rustls_error`. Variants of `rustls::Error`, `CertificateError`,
  and `PeerIncompatible` are matched directly. Debug is used ONLY for
  surfacing variant names in user-facing messages, never for routing.
- #21: Fail-closed — unknown `CertificateError` variants default to
  `TlsError::CertificateValidationFailed` (not `ConnectionFailed`), so a
  future rustls bump that adds a cert variant never silently demotes
  validation failure to a retry hint (CWE-754 / CWE-392). Dead `san`
  substring branch removed with the closure rewrite.
- #22: `--allow-invalid-certificate` now requires a second-confirmation
  flag `--i-understand-this-is-insecure` (or env `GOLD_DIGGER_ALLOW_INVALID=1`).
  Without the companion, `to_tls_config()` returns
  `TlsError::MutuallyExclusiveFlags` → exit 2 (CWE-295 / CWE-296).
  `display_security_warnings` now sleeps 3 seconds after the `[DANGER]`
  banner in `AcceptInvalid` mode (skipped under `cfg(test)`).
- #45: Move `from_tls_options` adapter from `tls.rs` to `cli.rs` as
  `TlsOptions::to_tls_config`. `tls` module now depends only on primitive
  types (`bool`, `Option<PathBuf>`), enabling future extraction.
- #28: Add `tests/tls_classifier_rcgen.rs` (14 tests) using `rcgen` to
  generate real PEM material and drive the classifier through the same
  paths the production Pool::new closure takes. Includes the fail-closed
  assertion for unknown cert variants and redaction-hygiene checks.
- #46: Delete the 18 `TlsError::*` constructor helper methods. All
  callers (src + tests) migrated to direct struct init.

Also updates 3 insta snapshots for the new `--i-understand-this-is-insecure`
flag and propagates the companion flag through 28 integration test sites.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
The monolithic `src/tls.rs` mixed the error taxonomy, the rustls->TlsError
classifier, the config DTO, the CA-file loader, and the Pool builder in
one ~1900-line file with ~1100 lines of inline tests. Per the project's
file-size guideline (>=500 preferred, 1000 hard limit), split into a
directory module where each file has a single concern:

- src/tls/mod.rs       (43 lines) - //! doc + pub re-exports
- src/tls/error.rs    (487 lines) - TlsError enum + ctors + is_* helpers
- src/tls/classifier.rs (285 lines) - rustls::Error -> TlsError mapping
- src/tls/config.rs   (812 lines) - TlsConfig, TlsValidationMode, adapter
- src/tls/ca.rs       (124 lines) - CA PEM loading (re-exported as cert_utils)
- src/tls/pool.rs     (266 lines) - create_tls_connection + redact_url

The public API is preserved verbatim via `pub use` in `mod.rs`, so the
existing `gold_digger::tls::{TlsError, TlsConfig, TlsValidationMode,
create_tls_connection, redact_url, cert_utils, tls_config_from_url}`
import paths used by src/main.rs, src/cli.rs, src/exit.rs, and the
tests/ suite continue to resolve unchanged. Tests (inline #[cfg(test)])
moved to the submodule that owns the code under test; no test coverage
was lost.

No behavioural changes:
- All 45 inline tls:: unit tests pass.
- All external tls integration tests pass (tls_classifier_rcgen,
  tls_config_unit_tests, tls_backward_compatibility, tls_variants_test).
- Full `cargo nextest run` reports 552/552 passed, 13 skipped.
- `cargo clippy --all-targets -- -D warnings` is clean.
- `cargo deny check` reports advisories/bans/licenses/sources all ok.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
…s (P1-E)

Resolves 5 P1 todos in the format/output cleanup cluster. Net effect:
one `json::write` entry point, one free-function contract across all
three writers, and a fail-fast OutputFormat resolver that refuses to
guess on unknown extensions.

- #8 Delete JSON string-inference path. `JsonWriter::write_row` used
  to run every value through `parse::<u64>() → parse::<i64>() →
  parse::<f64>() → to_lowercase()=="true"` and coerce matches into JSON
  numbers/bools. That corrupted ZIP codes (`"00123"` → `123`), string
  flags (`"TRUE"` → `true`), phone numbers, country codes, and any
  identifier that happened to parse as a number. The typed-JSON path
  via `TypeTransformer::row_to_json` is now the only path. Pinned
  snapshots in `tests/tests/snapshots/json_null_handling.snap` migrated
  (NULLs are now `serde_json::Value::Null`, not `""`). Added two
  regression tests in `src/json.rs` that pin the new semantics:
  leading-zero strings and mixed-case `"TRUE"`/`"FALSE"` stay strings.

- #15 Delete 3-of-4 JSON write entry points. Removed `JsonWriter`
  struct, `json::write()`, `json::write_with_pretty()`, and the
  separate `write_typed(Vec<mysql::Row>, ...)` re-export. The sole
  remaining entry point is `json::write(maps, output, pretty)` which
  mirrors `csv::write` / `tab::write` for contract symmetry. The
  convert-before-create-file discipline is preserved: the caller
  validates all rows via `TypeTransformer::row_to_json` before the
  output file is opened, so a type error on row N never leaves a
  truncated file behind. src/json.rs shrank from 461 → 176 lines.

- #20 Delete the unused `FormatWriter` trait. It was implemented only
  by the now-deleted `JsonWriter`, its signature (`Item = &[String]`)
  did not match the documented `AsRef<str>` contract in AGENTS.md, and
  `main.rs::write_output` never dispatched through it. Picked the
  simpler "Option A" from the todo: single free `write()` per module,
  no trait abstraction. Updated AGENTS.md writer-contract paragraph to
  match.

- #19 `OutputFormat::from_extension` returns `Option<Self>`. Previous
  signature silently fell back to TSV for `.xml`, `.yaml`, `.data`,
  uppercase `.CSV`, or a missing extension — violating fail-fast, and
  combining poorly with the symlink/traversal work on #23. New impl
  lowercases the extension for case-insensitive matching and returns
  `None` on unknown/missing extensions. `main.rs::write_output` now
  raises `GoldDiggerError::Config` with a remediation hint
  ("Cannot infer output format ... Pass --format <csv|json|tsv> ...")
  instead of silently emitting TSV. Added six unit tests in
  `src/cli.rs` covering lowercase, uppercase, missing, and unknown
  extensions.

- #11 Remove the vestigial `json` and `csv` feature flags. Both were
  empty marker features (`json = []`, `csv = []`) in the default set;
  the `csv` crate was always linked, the `csv`/`json` modules were
  compiled unconditionally, and `--no-default-features --features
  "json csv"` produced a binary byte-for-byte identical to the default
  build. README and docs advertised a non-existent attack-surface
  reduction. Dropped the flag definitions from Cargo.toml, removed the
  `#[cfg(feature = "json")]` and `#[cfg(feature = "csv")]` markers
  from `src/main.rs`, `src/lib.rs`, and `dump_configuration`, and
  updated every occurrence of `--features "json csv
  additional_mysql_types verbose"` in justfile to `--features
  "additional_mysql_types verbose"`. Release binary size is unchanged
  (3,250,832 bytes) — confirming the flags gated nothing.

Tests: 565/565 pass via `just ci-check`; advisories/bans/licenses/
sources OK; rustls-only dep tree preserved. Snapshot migration
`tests/tests/snapshots/json_null_handling.snap` is the only touched
snapshot file; it pins the new null-as-JSON-null contract.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Implements the P1-F path-safety batch (todos #23, #24, #30):

- #23: validate `--query-file` before reading. The path is canonicalised
  via `std::fs::canonicalize` (resolves `..` and symlinks), the target
  extension is checked against a case-insensitive deny-list (`.exe`,
  `.dll`, `.so`, `.dylib`, `.bin`, `.bat`, `.cmd`, `.com` refused; `.sql`,
  `.txt`, or missing allowed), and files larger than 10 MiB are refused
  with a configuration error to cap DoS risk. Broken symlinks or missing
  files surface as `GoldDiggerError::Io` (exit 5); policy rejections map
  to `GoldDiggerError::Config` (exit 2).

- #24: harden output file creation. All three `File::create` call sites
  in `write_output` now go through `create_output_file`, which on Unix
  uses `OpenOptions::create_new(true).custom_flags(libc::O_NOFOLLOW)
  .mode(0o600)`. Pre-existing targets are refused with a configuration
  error pointing at the new `--force` flag; symlinks at the target are
  refused regardless of `--force` (O_NOFOLLOW). `libc` is added as a
  unix-only direct dep for `O_NOFOLLOW`. The Windows path falls back to
  `create_new(true)` / `create(true) + truncate(true)` under `--force`.

- #30: add `tests/path_safety.rs` with 10 tests covering the `.exe`
  extension reject, uppercase `.SQL` accept, missing-extension accept,
  size-cap reject, symlink-after-canonicalise reject, broken-symlink I/O
  error, pre-existing-output reject, `--force` bypass, Unix symlink
  output reject via O_NOFOLLOW, and `--force` on missing output path.

Integration-test common helpers pass `--force` at each `--output` site
because they pre-seed the path with `NamedTempFile`. SECURITY.md is
updated to document the new guards for both `--query-file` and
`--output`. Help-text snapshots are regenerated for the new `--force`
flag and updated `--query-file` prose.

`just ci-check` passes (575 tests; advisories/bans/licenses/sources ok).

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
… (P1-H)

Resolve the three P1-H UX todos in a single batch so the new logging,
colour, and progress code share one helper module and one set of
NO_COLOR / TTY / quiet detection rules.

#161 (color output) — add owo-colors with the supports-colors feature.
Introduce `logging::warn_banner` and `logging::danger_banner` that wrap
`[WARNING]` / `[DANGER]` text in yellow / red+bold ANSI codes when
stderr is a TTY AND `NO_COLOR` is unset. TLS `display_security_warnings`
now renders coloured banners for `SkipHostnameVerification` and
`AcceptInvalid`; output is plain when redirected to a file or when the
caller sets `NO_COLOR=1`.

#162 (progress indicators) — add indicatif. Connect / query phases use
indeterminate spinners; the write phase uses a bounded bar with row
count and ETA. `logging::make_progress` returns `ProgressBar::hidden()`
when `--quiet` is set, when stderr is not a TTY, or when `NO_COLOR` is
set, so callers don't branch on whether progress is visible. Spinners
finish-and-clear before tracing logs fire, keeping stderr greppable.

#163 (tracing-subscriber) — add tracing + tracing-subscriber. Initialise
a compact, plain-text stderr subscriber in `main()` before any work,
with level filtering derived from `cli.verbose` / `cli.quiet`:
`--quiet`→error, default→warn, `-v`→info, `-vv`→debug, `-vvv+`→trace.
`RUST_LOG` overrides for ad-hoc debugging. Replace all four `eprintln!`
sites in `src/main.rs` with `tracing::info!` (status) /
`tracing::error!` (failures), and migrate the remaining `eprintln!`
calls in `src/exit.rs` and `src/tls/{config,pool}.rs` to the same set
of macros so `--quiet` actually suppresses everything except errors.

Credential redaction is unchanged: `redact_sql_error` still scrubs
`mysql::Error` strings before they reach `tracing::error!`. The
subscriber does not inspect field contents — that's the caller's
responsibility, documented in the new module header.

cargo-deny: add `supports-color` (2.x ↔ 3.x divergence inside
owo-colors 4.3) and `windows-sys` (0.52 ↔ 0.61 divergence across the
new tracing / indicatif / clap subtrees) to `bans.skip` with explicit
`reason = ...` strings tracking the upstream consolidation status.

Snapshots: tls_cli_integration error snapshots now include the `ERROR`
level prefix that tracing-subscriber prepends. Updated via
`INSTA_UPDATE=always cargo test --test tls_cli_integration`.

Test coverage: 5 new unit tests in `src/logging.rs` covering NO_COLOR
gating (set / empty / unset), `--quiet` suppressing progress, NO_COLOR
suppressing progress, and `init_tracing` idempotence. Existing
580-test suite continues to pass under `just ci-check`.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Splits `src/main.rs` (~750 lines mixing CLI dispatch, config resolution,
TLS pool creation, query execution, output dispatch, completion
generation, and test harness) into focused modules:

- `src/main.rs` (~55 lines): parse CLI, init crypto + tracing, dispatch
  subcommands / `--dump-config`, hand off to `run::run(cli)`.
- `src/run.rs`: query-execution pipeline (resolve → connect → query →
  write) + MySQL-error-code mapper. Calls `config`, `connection`,
  `output`.
- `src/config.rs`: `resolve_database_url`, `resolve_database_query`,
  `resolve_output_file`, `dump_configuration`, `validate_query_file_path`
  and its size/extension constants.
- `src/connection.rs`: `create_database_connection` (TLS-aware pool
  builder).
- `src/completion.rs`: `generate_completion` for shell completions.
- `src/output.rs`: `write_output`, `create_output_file`, open-error
  classifier.

Unit tests moved alongside their subject (17 config, 3 connection, 4
completion = 24 tests). Each new module is published as `pub mod` under
`src/lib.rs` so future test code can reach them directly. No public-
binary behaviour changes: same CLI, same exit codes, same output
formats, same error messages.

Refs #14 (P1-K batch).

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Replace the eager `conn.query::<Row,_>` path (which materialised the full
result set into `Vec<mysql::Row>` before writing a single byte) with a
streaming pipeline that feeds `conn.query_iter` rows directly into a
format-specific `RowSink`. Peak memory is now linear in the column count,
not the row count — the previous 16 GB ceiling at ~2.3M JSON / ~3.5M CSV
rows no longer applies.

Pipeline shape
--------------
- New `src/sink.rs` defines the `RowSink` trait (`on_headers` / `on_row`
  / `finalize`) plus three implementations: `DelimitedSink` (CSV and
  TSV share plumbing, differ only in delimiter) and `JsonSink`.
- Each sink writes to `<output>.tmp` and renames on
  `finalize()`-success. `Drop` cleans up the `.tmp` on any failure,
  preserving the atomic-output invariant — a type-conversion error on
  row N now leaves **no** stale partial output behind.
- `src/run.rs::stream_query` opens `query_iter`, snapshots the column
  metadata once, builds the sink lazily (post-parse, pre-rows) and
  pushes rows into it. The progress indicator is now an indeterminate
  spinner that updates its message every 1000 rows (we can't know the
  total up-front with streaming).
- `src/output.rs` exposes `build_sink` for the live path and keeps
  `write_output(Vec<Row>, …)` for snapshot tests that feed in-memory
  rows. `write_output` now drives the same sink plumbing instead of
  the old `rows_to_strings` → `csv::write` helpers.

Error semantics
---------------
- Type-conversion failures are wrapped as `GoldDiggerError::Query`
  inside `sink::wrap_conversion_error` so exit code 4 stays stable
  even if the upstream `TypeTransformer` message text is refactored.
- `mysql::Error` from row fetch (mid-stream network failure etc.)
  flows through the same `map_query_error` + `redact_sql_error` path
  as the initial query error, so credentials still cannot leak.
- `--allow-empty` semantics are preserved: `on_headers` fires up-front
  so an empty stream can still be finalized to a valid envelope
  (JSON `{"data":[]}`, CSV/TSV header-only).

Pretty-print & backward-compat
------------------------------
- `--pretty` JSON serialises each row with `to_writer_pretty` (per-row
  indentation). The envelope `{"data":[` / `]}` markers stay compact;
  a separate polish pass can add newlines between rows if the UX
  warrants it.
- The public `rows_to_strings` function and the `write` free-fns in
  `csv.rs` / `json.rs` / `tab.rs` are preserved — they are unchanged
  and the existing snapshot tests still exercise them.

Benchmark note
--------------
- `benches/memory_ceiling.rs` is now a *historical* reference for the
  old `rows_to_strings` path; the docstring is updated to say so. A
  follow-up can either retire it or rewrite it to measure the streaming
  pipeline end-to-end.

Tests
-----
- 8 new unit tests in `src/sink.rs` cover header/row writes, empty-
  result envelopes for JSON, failure cleanup (`.tmp` removed, target
  missing), the "refuse existing file without --force" guard, the
  `--force` overwrite path, and exit-code routing for conversion
  errors.
- `just ci-check` passes: 588 tests run, 588 passed (1 slow),
  0 skipped; advisories/bans/licenses/sources ok.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Removes the second-confirmation gate added by P1-D for
--allow-invalid-certificate (todo #22). The companion flag
(--i-understand-this-is-insecure) and GOLD_DIGGER_ALLOW_INVALID env
gate are paternalistic — when a user explicitly passes a CLI flag, the
binary should honour it. Stderr [DANGER] warnings remain (and obey
--quiet / NO_COLOR / TTY detection); the 3-second pre-connect sleep is
gone. Audit-time grep on deployment manifests is the right enforcement
point, not binary-side structural overrides.

Drops:
- `i_understand_this_is_insecure` field from `cli::TlsOptions`
- `GOLD_DIGGER_ALLOW_INVALID` env var
- gating logic in `cli::TlsOptions::to_tls_config` (always proceeds
  with the user's choice)
- `std::thread::sleep(3s)` in `tls::config::display_security_warnings`
- doc trails in `connection.rs` referencing the gate
- 28 `--i-understand-this-is-insecure` arg additions in
  `tests/integration/data_types.rs`
- 13 `i_understand_this_is_insecure: ...` struct-init lines in
  `tests/tls_backward_compatibility.rs` and `tests/tls_config_unit_tests.rs`
- the `test_cli_allow_invalid_certificate_requires_companion_flag`
  negative test (no longer applicable)

Snapshot updates: cli_testing_example, tls_backward_compatibility,
tls_cli_integration, and tls_config_unit_tests help/error snapshots
regenerated to drop the removed flag from --help output.

Also lands the P2-DOCS batch (14 todos):
- AGENTS.md "Requirements Gap Analysis" reflects current state (F001-3,
  F005, F007, F008, F010 all closed); no longer references gitignored
  paths.
- CHANGELOG [Unreleased] now lists CLI-first config, typed exit codes,
  streaming output, --output / --query-file safety guards, structured
  logging, coloured stderr, progress indicators, new CI workflows.
- src/lib.rs: crate-level //! doc was already present; expanded
  init_crypto_provider rustdoc to document the silent-error-swallow
  trade-off; expanded get_required_env with a doctest example;
  documented case sensitivity gotcha on get_extension_from_filename.
- src/exit.rs: map_error_to_exit_code rustdoc warns that error-message
  text is part of the public API in the substring-fallback path.
- src/type_transformer.rs: added per-format usage table to
  TypeTransformer rustdoc.
- src/tls/pool.rs: expanded create_tls_connection rustdoc with full
  Arguments / Returns / Errors / Example sections.
- src/utils.rs: strengthened redact_sql_error doctest to assert URL
  userinfo replacement + idempotence.
- src/cli.rs: TlsOptions field rustdocs now classify the residual
  attack surface for tls_ca_file (CA-key compromise) and
  insecure_skip_hostname_verify (CWE-297).
- README.md: cross-link to tests/README.md and benches/README.md.

just ci-check: 587/587 tests pass; advisories/bans/licenses/sources ok.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Resolves the P2 test cluster surfaced in the comprehensive code review.
Each finding either gains targeted coverage or is consolidated against
existing work landed in earlier P1 commits.

New test surface:
- tests/redact_sql_error_truth_table.rs: rstest one-row-per-pattern
  table for redact_sql_error, including idempotence and a non-ASCII
  (contraseña) sentinel. (#74)
- tests/additional_mysql_types.rs: feature-gated coverage of DECIMAL
  precision, BIGINT u64::MAX/i64 boundaries, TIME signed/days
  aggregation, DATETIME microseconds, and NULL convergence. (#76)
- tests/exit_code_1_no_rows.rs: Docker-gated E2E for empty-result exit
  contract -- default exits 1 with no output file committed,
  --allow-empty exits 0 with `{"data":[]}` JSON / header-only CSV.
  (#77)
- tests/init_crypto_provider_idempotency.rs: serial + multi-thread
  idempotence pin for the `Once`-guarded rustls bootstrap. (#81)
- tests/tls_danger_banner_snapshot.rs: insta snapshot of the
  `--allow-invalid-certificate` [DANGER] banner block, plus a positive
  assertion that --quiet does not suppress error-level lines. (#82)
- tests/stream_separation.rs: stdout/stderr separation contract for
  --help, --version, --dump-config, completion, missing-config error,
  unreachable-host error, and the DANGER banner path. (#176)
- tests/cli_testing_example.rs: insta-pinned exact clap mutex error
  for --verbose + --quiet. (#78)

Bench:
- benches/memory_ceiling.rs: heavy 1M/2M/3M-row sweep gated on
  `IGNORE_BENCHES=0`; documents the legacy buffered curve so streaming
  (F007) regressions remain visible. (#83)

Snapshot path normalization (#79):
- Move the 40 snapshots under `tests/tests/snapshots/` to
  `tests/snapshots/` and update with_settings! to use
  `snapshot_path => "snapshots"` so a future test-file move no longer
  recreates the doubled path.

Already-resolved todos deleted:
- #75 (cross-redactor fidelity table) -- subsumed by P1-C
  (commit 549f8e0) and existing tests/dump_config_redaction.rs.
- #80 (migrate JSON snapshots from json::write to write_json_maps) --
  subsumed by P1-E (commit bf7ac78).
- #84 (.failure() -> .code(N) audit) -- subsumed by the P1-J sweep
  (no .failure() calls remain in tests/).
- #55 (atomic CSV/TSV temp-file rename) -- subsumed by F007 streaming
  sink (commit f052c46) which uses `.tmp + rename` across all formats.

Deferred:
- #85 (delete tls_integration_old.rs): file is referenced by
  .github/workflows/ci.yml; deletion needs a paired CI edit which is
  out of scope for this batch.
- #174 (consolidate TLS test corpus): 6-8 hour merge.
- #175 (split data_types.rs 5151 LOC): 8-10 hour mechanical refactor.

Verified via `just ci-check` (642 tests, 0 failures, 0 clippy warnings).

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Both .github/ci-performance-config.yml and
.github/error-reporting-config.yml are not referenced by any workflow,
issue template, or tool config -- leftover files from a discontinued
tool. Removing them reduces noise in the repo root.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
CLI hardening (todos #154, #177, #152, #179):
- Cli, Commands, Shell, OutputFormat now derive Debug for inspectability
- Commands, Shell, OutputFormat marked #[non_exhaustive] for downstream
  semver future-proofing (new variants no longer require a major bump)
- Shell and OutputFormat now derive Copy (cheap pass-by-value, removes
  needless .clone() calls -- output.rs updated to take advantage)
- Cli root #[command(...)] now exposes author, long_about, after_help
  with EXAMPLES section -- richer --help output for users

Documentation (todo #142):
- TypeTransformer::value_to_string and value_to_json gain doctest-tested
  Example sections so docs.rs shows runnable samples

Snapshot tests refreshed for the new --help output.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Release docs (#143, #156):
- RELEASING.md: correct '6 platforms' -> '7 target triples' to match
  dist-workspace.toml (macOS arm64/x64, Linux arm64/x64-gnu/x64-musl,
  Windows arm64/x64).
- RELEASING.md: add explicit `grep -rn 'v0\.[0-9]'` step to the
  pre-release checklist so version drift in AGENTS.md/CLAUDE.md/README.md
  is caught before tagging.

Documentation cross-links (#146):
- docs/src/development/api-reference.md: add canonical docs.rs link so
  external readers reach the published rustdoc when the local mdBook
  copy is unavailable.

Markdown hygiene (#141):
- AGENTS.md: replace four-backtick fence with a regular triple-backtick
  rust block + indented inner example. Renders correctly in older
  GitHub UIs and through mdformat.

Snapshot workflow (#160):
- justfile: add `insta-review` recipe (`INSTA_UPDATE=always cargo test`).
- CONTRIBUTING.md: document the snapshot regen flow and call out that
  `cargo insta review --accept` is NOT the supported command here.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Replace `*.snap binary` with `*.snap text eol=lf -diff` so insta
snapshot files stay byte-identical across macOS/Linux/Windows checkouts.
The previous `binary` attribute hid line-ending churn but still let
Windows checkouts produce CRLF-encoded snapshots that diverged from
Linux/macOS golden output. Diffing remains intentionally suppressed --
contributors review snapshot changes via `just insta-review` (todo #160)
rather than git diff.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Resolve P2 src-cleanup / refactor todos. Compiles together with the
in-flight #68 (env snapshot), #169 (lazy crypto init), and #177
(non-exhaustive Commands wildcard) work because the underlying tree
needed all three to compile cleanly after 77a448c.

Resolved (mine):
* #53/#60 — TlsError::from_rustls_error: switch user-facing messages
  to Display where rustls supports it; document Debug fallback for
  variants that don't (PeerMisbehaved, AlertDescription); add explicit
  fallthrough comment for the non_exhaustive rustls::Error.
* #54 — new src/mysql_errors.rs with named ER_*/CR_* constants;
  classify_mysql_error_code helper extracts the error-code → message
  table out of map_query_error so it's grep-able and unit-tested.
* #56 — document rows_to_strings empty-input behaviour and direct
  callers to crate::sink::json_sink for JSON envelope guarantees.
* #58 — extract crate::delimited::write_delimited shared helper used
  by csv.rs and tab.rs; both writers collapse to one-line delegates.
* #59 — pub const OUTPUT_BUFFER_CAPACITY in lib.rs; csv/tab/json/sink
  all reference the canonical knob.
* #63 — rows_to_strings now delegates per-row conversion to
  TypeTransformer::row_to_strings so the column-level error message
  matches the streaming sink path.
* #65 — Opts::from_url error path routes the URL through
  utils::redact_url and the error string through redact_sql_error so
  malformed-URL errors no longer leak credentials (CWE-532).
* #66 — fail-closed regex validation: REDACTION_PATTERN_DEFS slice +
  EXPECTED_REDACTION_PATTERN_COUNT pin; debug builds panic on bad
  pattern, release builds tracing::error! and skip; new test asserts
  count.
* #69 — TypeTransformer::row_to_json_with_columns helper takes a
  pre-extracted Arc<str> column-name slice; JsonSink populates the
  list once in on_headers and reuses it per row.
* #70 — drop the outer BufWriter on csv.rs/tab.rs; the shared
  delimited helper sets WriterBuilder::buffer_capacity instead.
* #71 — add itoa/ryu deps; TypeTransformer::value_to_string and
  format_special_float use stack buffers for integer/float formatting.
* #72 — add cold-path note to map_error_to_exit_code so future
  readers don't micro-optimise the substring matcher.
* #102 — closed (already optimised by 407bd0f's HEX_CHARS table).
* #106 — migrate src/lib.rs INIT from std::sync::Once to OnceLock<()>
  for consistency with src/utils.rs.
* #100 — already done by #5 streaming pipeline (f052c46).
* #55 — already done: src/sink.rs uses <output>.tmp + atomic rename.
* #61 — deferred: collapsing exit_with_error/exit_success/exit_no_rows
  into a single helper hurts call-site readability; current shape
  already routes through map_error_to_exit_code.
* #107 — deferred: cosmetic cache resolved by typed classifier ([R-H3]).

Resolved (parallel agent #20, P2-DOCS / config / crypto-init batch):
* #68/S15 — EnvSnapshot reads DATABASE_* once at startup; downstream
  resolvers consume the snapshot.
* #169 — install rustls crypto provider lazily inside
  create_database_connection; main.rs no longer pays the cost on
  --help / completion / --dump-config.
* #177 — Commands #[non_exhaustive] wildcard arm + dump-config
  refactor through build_configuration_dump.

Test plan
* cargo check / cargo clippy --all-targets -- -D warnings: clean.
* cargo test --lib: 178 passed.
* cargo test (full): all integration + doctests pass.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Resolve four small P2/P3 todos surfaced by the comprehensive review:

* #113 -- audit `.github/zizmor.yml` ignores: drop the stale
  `cargo-dist.yml` entry (workflow no longer exists, deleted in be42557)
  and add `# reason:` comments documenting why each remaining ignore
  exists. `codeql.yml` is preserved because the workflow file ships in
  this branch.
* #121 -- extract `TLS_MODE_GROUP_ID` const for the clap mutex group
  shared by `--tls-ca-file`, `--insecure-skip-hostname-verify`, and
  `--allow-invalid-certificate`. Future renames are now a single edit.
* #139 -- normalise test prefix in
  `tests/type_transformer_snapshots.rs` from `snapshot_*` to
  `test_*`. The 27 functions all use explicit `assert_snapshot!`
  literals, so snapshot file names are unchanged. `grep "^fn test_"`
  now finds the entire test surface in one sweep.
* #147 -- README "Development Status" claimed "Advanced Testing ...
  planned" while ROADMAP marked v0.3.0 testing complete. README now
  matches ROADMAP and points readers to `tests/` and `benches/`.

`just check` passes (633 tests, 0 clippy warnings, fmt clean).

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
@dosubot dosubot Bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:XXL This PR changes 1000+ lines, ignoring generated files. labels Apr 25, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 97 out of 144 changed files in this pull request and generated 5 comments.

Comment thread .github/workflows/ci.yml
Comment thread tests/test_support/cli.rs Outdated
Comment thread Cargo.toml Outdated
Comment thread tests/exit_code_3_auth.rs
Comment thread tests/integration/common.rs Outdated
@coderabbitai coderabbitai Bot added documentation Improvements or additions to documentation dependencies Pull requests that update a dependency file github_actions Pull requests that update GitHub Actions code rust Pull requests that update Rust code category/security Security-related tasks category/ci-cd CI/CD pipeline and workflow tasks category/testing Testing framework and quality assurance category/docs Documentation improvements category/code-quality Code quality and linting improvements and removed type/chore Maintenance and infrastructure tasks labels Apr 25, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 25, 2026

Codecov Report

❌ Patch coverage is 91.42756% with 242 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/output.rs 49.29% 36 Missing ⚠️
src/tls/pool.rs 69.36% 34 Missing ⚠️
src/config.rs 93.81% 29 Missing ⚠️
src/tls/classifier.rs 84.65% 29 Missing ⚠️
src/logging.rs 70.83% 28 Missing ⚠️
src/tls/ca.rs 84.89% 21 Missing ⚠️
src/run.rs 85.60% 19 Missing ⚠️
src/tls/error.rs 95.25% 12 Missing ⚠️
src/connection.rs 76.74% 10 Missing ⚠️
src/sink.rs 98.21% 7 Missing ⚠️
... and 7 more

📢 Thoughts on this report? Let us know!

The json and csv features were removed (Cargo.toml notes them as
empty-marker flags that built byte-identical binaries) but several
references still referenced them:

- .github/workflows/ci.yml: 'test-features (minimal)' job and the
  coverage 'minimal features' step both passed --features 'json csv ...'
  which now errors with 'package does not contain these features'.
- README.md, AGENTS.md: minimal-build example commands.

Also fixes a parallel-test env-var race exposed by cargo llvm-cov in CI:
test_resolve_database_query_from_env, test_resolve_database_query_missing,
test_resolve_output_file_from_env, and test_resolve_output_file_missing
called Cli::parse_from outside their temp_env::with_var closure.
Clap's #[arg(env = ...)] reads the live env at parse time, so a sibling
test mutating the same var could leak its value into cli.output / cli.query
before the closure even started. Moving parse_from inside the closure
puts both the read and the write under temp_env's serial mutex.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
@coderabbitai coderabbitai Bot removed documentation Improvements or additions to documentation dependencies Pull requests that update a dependency file github_actions Pull requests that update GitHub Actions code rust Pull requests that update Rust code category/security Security-related tasks category/ci-cd CI/CD pipeline and workflow tasks category/testing Testing framework and quality assurance category/docs Documentation improvements category/code-quality Code quality and linting improvements labels Apr 25, 2026
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.

Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.

👉 Steps to fix this

Actionable comments posted: 23

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)
tests/integration/common.rs (1)

1-2015: 🧹 Nitpick | 🔵 Trivial

File exceeds recommended length guideline.

At 2015 lines, this file significantly exceeds the ≤1000 line (preferably ≤500) guideline. Consider splitting into separate modules:

  • cli.rs for GoldDiggerCli and command execution
  • parsing.rs for OutputParser and format validation
  • temp.rs for TempFileManager and TestIsolation
  • predicates.rs for CliPredicates and validation predicates

This would improve maintainability and reduce cognitive load when navigating the test utilities.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/integration/common.rs` around lines 1 - 2015, The file is too large
(2015 lines); split it into focused modules to improve maintainability by
extracting related types and functions: create cli.rs and move GoldDiggerCli
(and its impls: execute, execute_with_assertions, execute_with_timeout,
execute_raw, execute_with_output_validation, execute_with_snapshot,
redact_sensitive_output) and AssertCmdIntegration into a cli module; create
parsing.rs and move OutputParser, CsvParseResult/JsonParseResult/TsvParseResult,
and OutputValidator + related predicate structs (CsvContentPredicates,
JsonContentPredicates, TsvContentPredicates) into a parsing module; create
temp.rs and move TempFileManager, TestIsolation, TestEnvironment,
TempFileManager methods and TempFileManager Drop into a temp module; create
predicates.rs and move CliPredicates and related predicate helpers into it;
update the original tests/integration/common.rs to declare mod cli; mod parsing;
mod temp; mod predicates; re-export the moved symbols (pub use
crate::cli::GoldDiggerCli; etc.) and update use paths inside moved code to
reference the new module boundaries (e.g., references to OutputParser in cli ->
parsing::OutputParser or re-exported names), ensure ENV_VARS_TO_REMOVE and
TestCase/OutputFormat/GoldDiggerResult remain accessible (keep them in common.rs
or a small shared mod) and run cargo check to fix any import visibility issues.
tests/integration/tls_tests.rs (2)

40-47: ⚠️ Potential issue | 🟠 Major

std::process::exit(0) in test skip logic bypasses test framework.

Using std::process::exit(0) terminates the entire test process, not just the current test case. This can mask failures in other tests running in the same process and prevents proper test reporting.

Consider using a proper test-skip mechanism or restructuring with conditional compilation:

🛠️ Proposed fix using early return with explicit skip
 /// Skip test if Docker is not available
-fn skip_if_no_docker() {
+fn skip_if_no_docker() -> bool {
     if !is_docker_available() {
         println!("Skipping test: Docker not available");
-        // Use proper test skipping mechanism
-        std::process::exit(0); // Exit gracefully for skipped tests
+        return true;
     }
+    false
 }

Then update call sites to return early:

fn test_platform_certificate_store(#[case] db_flavor: &str) -> Result<()> {
    if skip_if_no_docker() {
        return Ok(());
    }
    // ... rest of test
}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/integration/tls_tests.rs` around lines 40 - 47, The helper
skip_if_no_docker currently calls std::process::exit(0), which aborts the whole
test run; change skip_if_no_docker to return a bool (e.g., fn
skip_if_no_docker() -> bool) or Result to indicate skipping instead of exiting,
remove the std::process::exit(0) call, and update callers (like
test_platform_certificate_store) to check skip_if_no_docker() and return early
(e.g., if skip_if_no_docker() { return Ok(()); }) so only the individual test is
skipped and the test framework continues reporting other tests.

639-754: ⚠️ Potential issue | 🔴 Critical

Add missing imports for CertificateValidator and CertificateLoader from the fixtures module.

The ephemeral_certificate_tests module uses CertificateValidator and CertificateLoader (defined in tests/fixtures/tls/mod.rs), but these types are not imported in tests/integration/tls_tests.rs. The use super::*; in the submodule only brings in what's available in the parent scope, so this will fail to compile.

Add to the imports in tests/integration/tls_tests.rs:

use crate::fixtures::tls::{CertificateValidator, CertificateLoader, EphemeralCertificate};

Or import them separately if preferred. Without this, all test functions in ephemeral_certificate_tests will fail with "cannot find type" errors.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/integration/tls_tests.rs` around lines 639 - 754, The tests in the
ephemeral_certificate_tests module reference CertificateValidator,
CertificateLoader, and EphemeralCertificate but do not import them, causing
"cannot find type" compile errors; add imports for these symbols from the
fixtures tls module (e.g., import CertificateValidator, CertificateLoader,
EphemeralCertificate from crate::fixtures::tls or equivalent) at the top of
tests/integration/tls_tests.rs so the submodule can resolve those types
referenced in test_ephemeral_certificate_generation,
test_certificate_loading_utilities, and test_certificate_validation_utilities.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/dependabot.yml:
- Around line 15-16: YAML lint fails due to spaces inside inline array brackets;
update the inline arrays (the keys patterns and update-types) to remove inner
spacing so they read like patterns: ["*"] and update-types: ["minor","patch"]
(also apply the same fix to the other occurrences mentioned for the file).
- Around line 30-39: The "security-sensitive" group aggregates multiple crates
into one PR, which contradicts the comment that each crate should be reviewed
and held back individually; update the Dependabot config so matching crates are
not batched by this group—either remove the "security-sensitive" group and
create distinct entries for each crate pattern (the listed "mysql*", "rustls*",
"clap*", "tokio*", "serde*") or change the group settings to disable aggregation
so each pattern produces its own PR (update the "security-sensitive" grouping
behavior referenced by the group name and the "patterns" list).

In @.github/workflows/benchmarks.yml:
- Around line 8-13: The YAML linter flags extra spaces inside branch list
brackets for the workflow triggers; update the bracketed branch arrays under the
on block (the pull_request and push entries) from "branches: [ main ]" to
"branches: [main]" to make formatting consistent and satisfy the linter while
leaving behavior unchanged.

In @.github/workflows/cargo-audit-pr.yml:
- Around line 10-12: The YAML lint fails due to bracket-spacing in the
workflow's branch lists; update the two occurrences of "branches: [ main ]" to
remove the inner spaces so they read "branches: [main]" or, alternatively,
convert them to explicit YAML sequences like "branches:\n  - main"; update the
entries for both occurrences (the two "branches: [ main ]" lines) so the spacing
rule is satisfied.

In @.github/workflows/ci.yml:
- Around line 5-7: YAML lint fails due to spaces inside bracketed sequence
literals; update the branch and needs list syntaxes (the "branches:" entries and
any "needs:" list like the one around line 215) to remove inner spaces so
bracketed lists use the normalized form (e.g., change "[ main ]" and "[
test-tls, ... ]" to "[main]" and "[test-tls,...]" respectively) across the CI
workflow file to satisfy the bracket-spacing rule.

In @.github/workflows/codeql.yml:
- Around line 3-37: YAMLlint flags are caused by bracketed arrays with spaces
and the bare on key; update the workflow to use YAMLlint-safe styles by (1)
quoting the top-level key "on" (e.g., "on":) to avoid truthy-warning and (2)
normalize array syntax for keys like branches and language by removing spaces
inside brackets or converting to block lists (e.g., change branches: [ main ] to
branches: [main] or to branches: - main and matrix: language: - rust). Apply
these changes for all occurrences (branches, matrix.language, cron array) and
keep existing keys like concurrency, defaults, permissions, and jobs names
intact.

In @.github/workflows/coverage.yml:
- Around line 3-8: The bracket spacing in the workflow trigger arrays is
inconsistent; update the array literals under the on: push: branches and on:
pull_request: branches (and any similar arrays like workflow_dispatch if
present) to use compact bracket syntax (e.g., change "[ main ]" to "[main]") so
spacing matches the style used in benchmarks.yml.

In @.github/workflows/scorecard.yml:
- Line 15: The YAML lint error is caused by extra spaces inside the inline array
for the branches key; update the branches declaration (branches: [ "main" ]) to
remove the inner spaces so it becomes branches: ["main"] to satisfy YAMLlint's
inline-array spacing rule.

In @.github/workflows/security.yml:
- Around line 5-6: The YAML arrays "workflows: [ CI ]" and "types: [ completed
]" use inline spacing that fails YAMLlint; update them to the linter's
bracket-spacing style by normalizing spacing inside the brackets (e.g., remove
or adjust spaces around items so they match project lint rules) for the entries
currently shown as workflows and types in the security.yml workflow definition.

In `@AGENTS.md`:
- Around line 127-139: Replace the redundant phrase "comprehensive CLI
interface" in the Project Overview with a non-redundant alternative (e.g.,
"comprehensive CLI" or "comprehensive command-line interface"); locate the exact
text "comprehensive CLI interface" and update it so the description reads
cleanly (for example: "It features a comprehensive CLI" or "It features a
comprehensive command-line interface"), leaving surrounding lines like
"CLI-first with environment variable fallbacks using `clap`" unchanged.

In `@SECURITY.md`:
- Around line 101-104: The SECURITY.md claim that created output files are
always set to 0o600 conflicts with the repo policy to respect the system umask;
update the doc and/or implementation so they agree: either change the
SECURITY.md bullet ("0o600 permissions (Unix)") to state that files are created
with owner-only intent but the process respects the system umask (while still
enforcing O_CREAT | O_EXCL, O_NOFOLLOW and --force semantics), or change the
code that opens output files to apply the explicit 0o600 mode only after
clearing/applying the umask so the effective permissions respect the system
umask; ensure the wording references the same behaviors (O_CREAT | O_EXCL /
create_new, O_NOFOLLOW, --force, and the permission semantics) so documentation
and implementation match.

In `@src/completion.rs`:
- Around line 10-35: The public funcs generate_completion_to<W: Write> and
generate_completion are missing the required rustdoc sections; add standardized
/// docs for each including a short description plus # Arguments (documenting
shell and writer for generate_completion_to, shell for generate_completion), #
Returns (-> () ), and a # Example block (use no_run showing calling
generate_completion_to with a Vec<u8> writer and generate_completion with a
Shell variant), ensuring you reference the Shell type and Write bound in the
docs and keep the format consistent with other public APIs.

In `@src/connection.rs`:
- Around line 127-143: The tests test_create_database_connection and
test_create_database_connection_with_tls_options currently attempt real network
resolution against "nonexistent:3306", causing flaky DNS/timeouts; change them
to induce a deterministic parse/validation failure or move them to integration
tests. Specifically, update the test inputs used by create_database_connection
to a syntactically invalid DSN (e.g., a malformed URI) or otherwise trigger the
function's validation path so the call returns Err immediately instead of
attempting a network connect, or mark these as integration tests and remove them
from unit tests. Ensure the tests still assert result.is_err() and keep
references to TlsConfig and create_database_connection intact.

In `@src/delimited.rs`:
- Around line 54-65: The write and flush calls on the CSV writer lose context
because they use bare `?`; update the `wtr.write_record(row)?;` and
`wtr.flush()?;` calls to map errors into richer messages (e.g., include the
record index or contents and the operation) using `.map_err(...)` before the `?`
so failures from `wtr.write_record` and `wtr.flush` surface as contextualized
errors; target the `wtr` usage created via `WriterBuilder::new()` (variable
`wtr`) and ensure the returned error type matches the function's error
(wrap/convert as needed).

In `@src/logging.rs`:
- Around line 155-179: warn_banner and danger_banner double-allocate when colors
are supported because they call if_supports_color with a closure that returns a
String and then call .to_string() again; since you already check
stderr_supports_color(), remove the if_supports_color call and apply the style
directly to message (e.g., use message.yellow().to_string() for warn_banner and
message.red().bold().to_string() for danger_banner) and keep the else branch
returning message.to_owned(), referencing the warn_banner, danger_banner,
stderr_supports_color, and if_supports_color symbols to locate the code.

In `@src/tls/ca.rs`:
- Around line 123-161: The TOCTOU comes from checking ca_file_path.exists()
before File::open in load_ca_certificates; remove the exists() check and instead
attempt to open the file directly, mapping File::open errors into the
appropriate TlsError (e.g., map io::ErrorKind::NotFound to
TlsError::ca_file_not_found(ca_file_path.display().to_string()) and other I/O
errors to TlsError::invalid_ca_format with the error string) so behavior aligns
with CaFile::load and eliminates the race window.

In `@tests/additional_mysql_types.rs`:
- Around line 177-187: The test extended_types_null_paths_converge calls
TypeTransformer::value_to_string(&v).unwrap(); replace the .unwrap() with
.expect("value_to_string") to match the pattern used in other NULL tests (e.g.,
decimal_null_yields_empty_and_null), ensuring consistent failure messages;
update only the call in the test to use .expect with that string.
- Around line 116-154: Add a MySQL TIME maximum-boundary case to the cases array
in the time_boundaries_stable test: insert a tuple using Value::Time for the max
positive TIME (hours aggregated from days+h to 838:59:59) — e.g. (false, 34, 22,
59, 59, 0, "838:59:59.000000") — and optionally add the negative counterpart
(true, 34, 22, 59, 59, 0, "-838:59:59.000000"); then run the assertions that
call TypeTransformer::value_to_string and TypeTransformer::value_to_json to
ensure the transformer emits the canonical string for the extreme boundary.

In `@tests/credential_leak_sweep.rs`:
- Around line 46-53: LEAK_MARKERS currently lists common leak patterns but
misses "auth=" and "bearer="; update the LEAK_MARKERS constant to include
"auth=" and "bearer=" (e.g., add "auth=" and "bearer=" to the &[&str] array) so
tests detect those common credential patterns; ensure casing or trimming
behavior is consistent with existing entries in LEAK_MARKERS.

In `@tests/path_safety.rs`:
- Around line 150-202: The test function query_file_over_size_cap_rejected
declares a redundant variable size that is never passed into the assert!
message; either delete the unused size binding or actually use it in the
assertion by adding it to the format arguments and updating the format
placeholder (e.g., change "size_target={size}" to "{}" and append size as an
argument, or include size with a named format argument), ensuring the variable
name matches the placeholder or is removed entirely to avoid the unused-binding
warning.

In `@tests/stream_separation.rs`:
- Around line 122-137: The test connection_error_routes_to_stderr_only currently
only checks exit code and empty stdout but doesn't assert anything about stderr;
update the assertion chain on fresh_cmd() to also assert that stderr is
non-empty and contains the connection error diagnostic (e.g., use a predicate
like predicate::str::contains("connect") or "connection" or a suitable error
phrase) so the test fails if the diagnostic disappears; locate the assertion in
connection_error_routes_to_stderr_only and add the stderr(...) check after the
existing stdout(...) check.

In `@tests/test_support/cli.rs`:
- Around line 24-47: Add the required standardized doc sections to the two new
public helpers clean_cmd and clean_cmd_no_color: update each /// comment to
include an Arguments section (if none, state "None"), a Returns section
describing the returned assert_cmd::Command, and an Example section showing
minimal usage (example snippet invoking the function and running/asserting the
command); keep the existing descriptive text and ensure markdown formatting
(headings like "Arguments", "Returns", "Example") and triple-slash comments are
used for both functions.
- Around line 181-184: The test setup currently builds a Command inline and
manually removes ENV_VARS_TO_REMOVE; instead, refactor execute() to call the
existing clean_cmd() helper so command hygiene is centralized—locate the code
creating cmd in execute() and replace the manual env_remove loop with a call to
clean_cmd() (or have execute() delegate to clean_cmd() to obtain the sanitized
Command), ensuring you reference clean_cmd() and execute() so all env_scrub
logic lives in one place.

---

Outside diff comments:
In `@tests/integration/common.rs`:
- Around line 1-2015: The file is too large (2015 lines); split it into focused
modules to improve maintainability by extracting related types and functions:
create cli.rs and move GoldDiggerCli (and its impls: execute,
execute_with_assertions, execute_with_timeout, execute_raw,
execute_with_output_validation, execute_with_snapshot, redact_sensitive_output)
and AssertCmdIntegration into a cli module; create parsing.rs and move
OutputParser, CsvParseResult/JsonParseResult/TsvParseResult, and OutputValidator
+ related predicate structs (CsvContentPredicates, JsonContentPredicates,
TsvContentPredicates) into a parsing module; create temp.rs and move
TempFileManager, TestIsolation, TestEnvironment, TempFileManager methods and
TempFileManager Drop into a temp module; create predicates.rs and move
CliPredicates and related predicate helpers into it; update the original
tests/integration/common.rs to declare mod cli; mod parsing; mod temp; mod
predicates; re-export the moved symbols (pub use crate::cli::GoldDiggerCli;
etc.) and update use paths inside moved code to reference the new module
boundaries (e.g., references to OutputParser in cli -> parsing::OutputParser or
re-exported names), ensure ENV_VARS_TO_REMOVE and
TestCase/OutputFormat/GoldDiggerResult remain accessible (keep them in common.rs
or a small shared mod) and run cargo check to fix any import visibility issues.

In `@tests/integration/tls_tests.rs`:
- Around line 40-47: The helper skip_if_no_docker currently calls
std::process::exit(0), which aborts the whole test run; change skip_if_no_docker
to return a bool (e.g., fn skip_if_no_docker() -> bool) or Result to indicate
skipping instead of exiting, remove the std::process::exit(0) call, and update
callers (like test_platform_certificate_store) to check skip_if_no_docker() and
return early (e.g., if skip_if_no_docker() { return Ok(()); }) so only the
individual test is skipped and the test framework continues reporting other
tests.
- Around line 639-754: The tests in the ephemeral_certificate_tests module
reference CertificateValidator, CertificateLoader, and EphemeralCertificate but
do not import them, causing "cannot find type" compile errors; add imports for
these symbols from the fixtures tls module (e.g., import CertificateValidator,
CertificateLoader, EphemeralCertificate from crate::fixtures::tls or equivalent)
at the top of tests/integration/tls_tests.rs so the submodule can resolve those
types referenced in test_ephemeral_certificate_generation,
test_certificate_loading_utilities, and test_certificate_validation_utilities.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 89bb672f-fcbd-4d16-a2de-9da6b5746907

📥 Commits

Reviewing files that changed from the base of the PR and between ffd149a and 389996b.

⛔ Files ignored due to path filters (61)
  • .cargo/audit.toml is excluded by none and included by none
  • .gitattributes is excluded by none and included by none
  • .github/workflows/release.yml is excluded by !.github/workflows/release.yml and included by .github/**
  • Cargo.lock is excluded by !**/*.lock and included by none
  • benches/README.md is excluded by none and included by none
  • benches/memory_ceiling.rs is excluded by none and included by none
  • benches/memory_usage.rs is excluded by none and included by none
  • benches/output_formats.rs is excluded by none and included by none
  • docs/solutions/best-practices/drop-safe-atomic-file-output-rust-cli-2026-04-25.md is excluded by none and included by none
  • docs/solutions/best-practices/front-load-validation-with-resolved-config-type-2026-04-25.md is excluded by none and included by none
  • docs/solutions/best-practices/replace-substring-error-classifiers-with-typed-enums-2026-04-25.md is excluded by none and included by none
  • docs/solutions/best-practices/unified-credential-redaction-rust-binary-2026-04-25.md is excluded by none and included by none
  • mise.lock is excluded by !**/*.lock and included by none
  • tests/snapshots/cli_testing_example__help_output.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/cli_testing_example__verbose_quiet_mutex_error.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/csv_empty_result_set.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/csv_escaping_quotes_and_commas.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/csv_newlines.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/csv_null_values.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/csv_standard_data.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/json_empty_result_set.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/json_large_integers.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/json_null_handling.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/json_pretty_printed.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/json_standard_data.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/tls_backward_compatibility__integration_compatibility_tests__cli_help_output.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/tls_cli_integration__tls_cli_flag_tests__invalid_ca_file_content_error.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/tls_cli_integration__tls_cli_flag_tests__nonexistent_ca_file_error.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/tls_cli_integration__tls_cli_flag_tests__tls_help_options.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/tls_danger_banner_snapshot__allow_invalid_certificate_danger_banner.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/tsv_null_conversion.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/tsv_special_characters.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/tsv_standard_data.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_json_bytes_valid_utf8.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_json_date.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_json_datetime_with_microseconds.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_json_double.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_json_float_infinity.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_json_float_nan.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_json_int.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_json_invalid_date_error.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_json_large_binary.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_json_null.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_json_time.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_json_uint_max.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_string_date.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_string_datetime.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_string_double.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_string_float.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_string_infinity.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_string_int.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_string_invalid_utf8.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_string_large_binary.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_string_nan.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_string_neg_infinity.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_string_null.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_string_time.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_string_time_with_days.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_string_uint.snap is excluded by !**/*.snap and included by tests/**
  • tests/snapshots/value_to_string_utf8_bytes.snap is excluded by !**/*.snap and included by tests/**
  • tests/tests/snapshots/json_null_handling.snap is excluded by !**/*.snap and included by tests/**
📒 Files selected for processing (83)
  • .cursor/rules/rust-best-practices.mdc
  • .github/ci-performance-config.yml
  • .github/dependabot.yml
  • .github/error-reporting-config.yml
  • .github/workflows/benchmarks.yml
  • .github/workflows/cargo-audit-pr.yml
  • .github/workflows/ci.yml
  • .github/workflows/codeql.yml
  • .github/workflows/copilot-setup-steps.yml
  • .github/workflows/coverage.yml
  • .github/workflows/scorecard.yml
  • .github/workflows/security.yml
  • .github/zizmor.yml
  • AGENTS.md
  • CHANGELOG.md
  • CONTRIBUTING.md
  • Cargo.toml
  • README.md
  • RELEASING.md
  • SECURITY.md
  • deny.toml
  • dist-workspace.toml
  • docs/src/development/api-reference.md
  • docs/src/development/architecture.md
  • docs/src/troubleshooting/type-errors.md
  • docs/src/usage/configuration.md
  • justfile
  • mise.toml
  • src/cli.rs
  • src/completion.rs
  • src/config.rs
  • src/connection.rs
  • src/csv.rs
  • src/delimited.rs
  • src/exit.rs
  • src/json.rs
  • src/lib.rs
  • src/logging.rs
  • src/main.rs
  • src/mysql_errors.rs
  • src/output.rs
  • src/run.rs
  • src/sink.rs
  • src/tab.rs
  • src/tls.rs
  • src/tls/ca.rs
  • src/tls/classifier.rs
  • src/tls/config.rs
  • src/tls/error.rs
  • src/tls/mod.rs
  • src/tls/pool.rs
  • src/type_transformer.rs
  • src/utils.rs
  • tessl.json
  • tests/additional_mysql_types.rs
  • tests/atomic_output.rs
  • tests/cli_testing_example.rs
  • tests/credential_leak_regression.rs
  • tests/credential_leak_sweep.rs
  • tests/dump_config_redaction.rs
  • tests/exit_code_1_no_rows.rs
  • tests/exit_code_3_auth.rs
  • tests/exit_code_4_query.rs
  • tests/exit_code_5_io.rs
  • tests/exit_codes.rs
  • tests/init_crypto_provider_idempotency.rs
  • tests/integration/common.rs
  • tests/integration/tls_tests.rs
  • tests/lib.rs
  • tests/output_format_snapshots.rs
  • tests/path_safety.rs
  • tests/redact_sql_error_truth_table.rs
  • tests/stream_separation.rs
  • tests/test_support/cli.rs
  • tests/tls_backward_compatibility.rs
  • tests/tls_classifier_rcgen.rs
  • tests/tls_cli_integration.rs
  • tests/tls_config_unit_tests.rs
  • tests/tls_danger_banner_snapshot.rs
  • tests/tls_integration_old.rs
  • tests/tls_variants_test.rs
  • tests/type_safety.rs
  • tests/type_transformer_snapshots.rs
💤 Files with no reviewable changes (3)
  • .github/error-reporting-config.yml
  • .github/ci-performance-config.yml
  • src/tls.rs

Comment thread .github/dependabot.yml Outdated
Comment thread .github/dependabot.yml Outdated
Comment thread .github/workflows/benchmarks.yml
Comment thread .github/workflows/cargo-audit-pr.yml Outdated
Comment thread .github/workflows/ci.yml Outdated
Comment thread tests/credential_leak_sweep.rs
Comment thread tests/path_safety.rs
Comment thread tests/stream_separation.rs Outdated
Comment thread tests/test_support/cli.rs
Comment thread tests/test_support/cli.rs Outdated
GitHub's Default Setup CodeQL is enabled at the repo level (visible as
the dynamic 'PR #N' workflow that already passes). The advanced
.github/workflows/codeql.yml has been failing on every PR commit with:

  Code Scanning could not process the submitted SARIF file:
  CodeQL analyses from advanced configurations cannot be processed
  when the default setup is enabled

The two cannot coexist. Default Setup provides equivalent (auto-pinned,
auto-maintained) coverage for the same language matrix.

Also drops the now-stale codeql.yml entry from .github/zizmor.yml's
unpinned-uses ignore list.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 96 out of 143 changed files in this pull request and generated 7 comments.

Comment thread .github/workflows/security.yml Outdated
Comment thread .github/workflows/coverage.yml Outdated
Comment thread .github/workflows/ci.yml Outdated
Comment thread tests/exit_code_3_auth.rs Outdated
Comment thread tests/exit_code_5_io.rs Outdated
Comment thread README.md
Comment thread tests/integration/common.rs Outdated
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.

Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.

👉 Steps to fix this

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/zizmor.yml:
- Around line 9-24: Update the .github/zizmor.yml suppressions so only
release.yml remains under unpinned-uses.ignore and remove docs.yml, ci.yml, and
security.yml from that list (they are SHA-pinned); also remove ci.yml from the
excessive-permissions suppression because its permissions are contents: read (no
write) and therefore should be evaluated normally; finally, adjust the
surrounding reason comment to state that only release.yml is intentionally
pinned by tag (v6.0.2) while the others are SHA-pinned and thus not suppressed.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 5fab9b9e-4901-4697-9dd3-bc43f8c62415

📥 Commits

Reviewing files that changed from the base of the PR and between 389996b and 5add972.

📒 Files selected for processing (1)
  • .github/zizmor.yml

Comment thread .github/zizmor.yml
Comment on lines +9 to 24
# reason: release.yml needs `contents: write` for tags/artifacts;
# docs.yml needs `pages: write` to publish mdBook; ci.yml's broad
# token surface is intentional for the matrix jobs.
- release.yml
- docs.yml
- ci.yml
unpinned-uses:
ignore:
# reason: third-party action versions are pinned by tag rather than
# SHA in these workflows; tracked separately in todo #013 (P1) for
# the release pipeline. The list intentionally drops the historical
# cargo-dist.yml entry — that workflow no longer exists (todo #113).
- release.yml
- cargo-dist.yml
- ci.yml
- codeql.yml
- docs.yml
- security.yml
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

workflows=(.github/workflows/release.yml .github/workflows/ci.yml .github/workflows/docs.yml .github/workflows/security.yml)

echo "== uses refs classification =="
for wf in "${workflows[@]}"; do
  echo "-- $wf --"
  rg -n '^\s*-\s*uses:\s*[^[:space:]]+@[^[:space:]]+' "$wf" | while IFS=: read -r ln txt; do
    ref="$(sed -E 's/.*@([^[:space:]#]+).*/\1/' <<<"$txt")"
    if [[ "$ref" =~ ^[0-9a-fA-F]{40}$ ]]; then kind="sha-pinned"; else kind="non-sha"; fi
    echo "Line $ln [$kind] $ref"
  done
done

echo
echo "== permissions blocks (root + jobs) =="
for wf in .github/workflows/ci.yml .github/workflows/docs.yml .github/workflows/release.yml; do
  echo "-- $wf --"
  rg -n -C2 '^\s*permissions:' "$wf"
done

Repository: EvilBit-Labs/gold_digger

Length of output: 1986


Remove SHA-pinned workflows from unpinned-uses suppressions and evaluate ci.yml in excessive-permissions.

ci.yml, docs.yml, and security.yml are all SHA-pinned, so they should not appear in unpinned-uses.ignore. Additionally, ci.yml has only contents: read permissions (no write scope), so its suppression under excessive-permissions should be removed.

Only release.yml should remain in unpinned-uses.ignore—it uses tag pins (v6.0.2) rather than SHAs. Update the reason comment to reflect this.

Proposed changes
 rules:
   excessive-permissions:
     ignore:
       # reason: release.yml needs `contents: write` for tags/artifacts;
-      # docs.yml needs `pages: write` to publish mdBook; ci.yml's broad
-      # token surface is intentional for the matrix jobs.
+      # docs.yml needs `pages: write` to publish mdBook.
       - release.yml
       - docs.yml
-      - ci.yml
   unpinned-uses:
     ignore:
-      # reason: third-party action versions are pinned by tag rather than
-      # SHA in these workflows; tracked separately in todo `#013` (P1) for
-      # the release pipeline. The list intentionally drops the historical
-      # cargo-dist.yml entry — that workflow no longer exists (todo `#113`).
+      # reason: release.yml is still tag-pinned; migrate to full SHAs in
+      # todo `#013` (P1). cargo-dist.yml no longer exists (todo `#113`).
       - release.yml
-      - ci.yml
-      - docs.yml
-      - security.yml
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# reason: release.yml needs `contents: write` for tags/artifacts;
# docs.yml needs `pages: write` to publish mdBook; ci.yml's broad
# token surface is intentional for the matrix jobs.
- release.yml
- docs.yml
- ci.yml
unpinned-uses:
ignore:
# reason: third-party action versions are pinned by tag rather than
# SHA in these workflows; tracked separately in todo #013 (P1) for
# the release pipeline. The list intentionally drops the historical
# cargo-dist.yml entry — that workflow no longer exists (todo #113).
- release.yml
- cargo-dist.yml
- ci.yml
- codeql.yml
- docs.yml
- security.yml
# reason: release.yml needs `contents: write` for tags/artifacts;
# docs.yml needs `pages: write` to publish mdBook.
- release.yml
- docs.yml
unpinned-uses:
ignore:
# reason: release.yml is still tag-pinned; migrate to full SHAs in
# todo `#013` (P1). cargo-dist.yml no longer exists (todo `#113`).
- release.yml
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/zizmor.yml around lines 9 - 24, Update the .github/zizmor.yml
suppressions so only release.yml remains under unpinned-uses.ignore and remove
docs.yml, ci.yml, and security.yml from that list (they are SHA-pinned); also
remove ci.yml from the excessive-permissions suppression because its permissions
are contents: read (no write) and therefore should be evaluated normally;
finally, adjust the surrounding reason comment to state that only release.yml is
intentionally pinned by tag (v6.0.2) while the others are SHA-pinned and thus
not suppressed.

YAML lint / workflow style:
- Normalize bracket spacing across .github/dependabot.yml, benchmarks.yml,
  cargo-audit-pr.yml, ci.yml, coverage.yml, scorecard.yml, security.yml,
  ci.yml needs: list (no more `[ main ]` etc.).
- Reflow split-line `run:` blocks in ci.yml minimal-features step,
  security.yml grype invocation, and coverage.yml llvm-cov to single
  lines so YAML plain-scalar folding is no longer load-bearing.
- Remove security-sensitive grouping in dependabot.yml so mysql/rustls/
  clap/tokio/serde updates ship as individual PRs (matches the comment's
  stated intent of per-crate review isolation).

Code/correctness:
- Cargo.toml: switch [profile.release] from panic=abort to panic=unwind.
  The atomic-output pipeline depends on Drop for .tmp cleanup
  (DelimitedSink/JsonSink/ProgressGuard); abort would silently leak
  partial files on panic.
- src/completion.rs: add Arguments + Example rustdoc sections to the
  public generate_completion / generate_completion_to functions.
- src/connection.rs: replace live-network 'nonexistent:3306' with
  127.0.0.1:1 (tcpmux, guaranteed unbound) in two unit tests so they
  don't depend on DNS or network egress.
- src/delimited.rs: wrap the per-row write and final flush in
  with_context() so failures report the row index instead of bare
  'broken pipe'.
- src/logging.rs: drop a double-allocation pattern in warn_banner /
  danger_banner -- inner closures now return borrowed Display wrappers
  so only the outer to_string allocates on the colored path.
- src/tls/ca.rs: port the canonicalize-first / open-after TOCTOU
  hardening from CaFile::load into legacy load_ca_certificates so both
  APIs offer the same guarantee. Tightened parameter type to &Path.
- AGENTS.md: drop redundant 'CLI interface' phrasing.

Test hardening:
- tests/test_support/cli.rs: GoldDiggerCommand::execute() now constructs
  via clean_cmd() instead of re-running cargo_bin_cmd! + env_remove,
  single-sourcing ENV_VARS_TO_REMOVE. Public helpers gained # Returns
  and # Example doc sections.
- tests/integration/common.rs: fresh_cmd() returns Command directly
  (cargo_bin_cmd! is infallible), and delegates to
  test_support::cli::clean_cmd() instead of duplicating the env list.
  Added 'mod test_support;' to the five integration-test binaries that
  depend on it.
- tests/exit_code_3_auth.rs, tests/exit_code_5_io.rs: replace hardcoded
  /tmp output paths with tempfile::tempdir() so tests are deterministic
  cross-platform and don't spuriously trip OutputExists (exit 2) when
  the expected failure mode is auth (3) or IO (5).
- tests/additional_mysql_types.rs: pin both ends of MySQL TIME range
  (838:59:59 / -838:59:59); replace .unwrap() with .expect() for a
  named NULL-path assertion.
- tests/credential_leak_sweep.rs: extend LEAK_MARKERS with auth=,
  bearer , and Bearer  so the sweep covers URL query / form fields and
  HTTP Authorization headers.
- tests/path_safety.rs: replace the unused size binding with named
  CAP_BYTES / EXPECTED_SIZE constants and an exact-size assert_eq so
  fixture drift is caught.
- tests/stream_separation.rs: assert that connection failures actually
  appear on stderr (not just absent from stdout) so a future logging
  refactor can't silently misroute the breadcrumb.
- tests/tls_classifier_rcgen.rs: drop unnecessary to_path_buf() calls
  after the load_ca_certificates signature tightened to &Path.

Validation: just ci-check -- 752/752 tests passed (1 slow), deny-check
clean.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.

Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.

👉 Steps to fix this

Actionable comments posted: 6

♻️ Duplicate comments (1)
.github/workflows/security.yml (1)

5-5: ⚠️ Potential issue | 🟡 Minor

Fix inline-array spacing on Line 5 to unblock YAMLlint.

workflows: [ CI ] still violates the bracket-spacing rule; use workflows: [CI].

Suggested fix
-        workflows: [ CI ]
+        workflows: [CI]
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/security.yml at line 5, The inline array spacing in the
YAML key "workflows" is violating YAMLlint's bracket-spacing rule; update the
value used in the workflow declaration from "workflows: [ CI ]" to remove the
spaces inside the brackets (i.e., "workflows: [CI]") so the inline array
conforms to the linter rule.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/workflows/coverage.yml:
- Around line 10-12: The workflow currently only sets permissions: contents:
read but still relies on the CODECOV_TOKEN secret; update the permissions block
to include id-token: write (e.g., add "id-token: write" alongside contents:
read), change the Codecov action invocation to use OIDC by setting use_oidc:
true and remove the CODECOV_TOKEN secret input (or guard its removal by
switching to the OIDC flow in the codecov action call), and ensure any
references to CODECOV_TOKEN in the job steps are deleted so the action
authenticates via OIDC instead of the long‑lived secret.

In `@src/logging.rs`:
- Around line 56-84: Add the missing documentation sections for the new public
logging helpers: init_tracing, stderr_supports_color, warn_banner,
danger_banner, and make_progress. For each function add /// Arguments describing
parameters, /// Returns describing the return value (or unit), and an ///
Example showing minimal usage (copyable snippet) consistent with existing crate
style; keep wording concise and include any side-effects (e.g., global
subscriber installation in init_tracing or terminal output for banners). Ensure
the doc comments are placed immediately above each pub fn and follow the
project's three-section requirement (Arguments, Returns, Example).

In `@src/tls/ca.rs`:
- Around line 8-12: Imports in src/tls/ca.rs are misordered; reorder them to
follow the coding guideline: list std imports first (e.g., std::fs::File,
std::io::BufReader, std::path::{Path, PathBuf}), then external crate imports
(e.g., rustls_pki_types::{CertificateDer, pem::PemObject}), and finally local
module imports (e.g., use super::error::TlsError), with a blank line between
each group to match the project's import ordering convention.

In `@tests/credential_leak_sweep.rs`:
- Around line 163-166: Replace the hardcoded host-dependent missing CA path
"/nonexistent/path/to/ca.pem" used in the .args([...]) call in
tests/credential_leak_sweep.rs with a deterministic tempdir-relative path (e.g.,
create a TempDir via tempfile::tempdir() or use std::env::temp_dir() and join a
non-existent filename) and pass that joined path to the "--tls-ca-file" argument
so the test fails for a missing file consistently across platforms
(Windows/Linux/macOS).

In `@tests/exit_code_3_auth.rs`:
- Around line 28-51: The test constructs the Command by manually removing env
vars (cargo::cargo_bin_cmd!("gold_digger") with multiple .env_remove calls),
which duplicates scrub logic and can miss vars like RUST_LOG honored by
src/logging.rs::init_tracing; replace this manual setup with the shared helper
tests/test_support/cli.rs::clean_cmd() so the test calls
clean_cmd(cargo::cargo_bin_cmd!("gold_digger")) (or the equivalent pattern used
elsewhere) and remove the explicit .env_remove calls to ensure all scrubbed envs
(including RUST_LOG) are consistently applied and keep exit-code assertions
unchanged.

In `@tests/test_support/cli.rs`:
- Around line 16-23: The tests currently fail to scrub RUST_LOG from spawned
test processes; update the shared test helper so clean_cmd() also removes
"RUST_LOG" from the environment (i.e., add "RUST_LOG" to the ENV_VARS_TO_REMOVE
constant and ensure the code in clean_cmd() iterates that list and calls
.env_remove(...) for each entry) so parent-shell RUST_LOG won't leak into test
subprocesses or logging.rs behavior.

---

Duplicate comments:
In @.github/workflows/security.yml:
- Line 5: The inline array spacing in the YAML key "workflows" is violating
YAMLlint's bracket-spacing rule; update the value used in the workflow
declaration from "workflows: [ CI ]" to remove the spaces inside the brackets
(i.e., "workflows: [CI]") so the inline array conforms to the linter rule.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 6ff5f849-d075-4ba8-9015-660922455ea3

📥 Commits

Reviewing files that changed from the base of the PR and between 5add972 and 72ba337.

📒 Files selected for processing (28)
  • .github/dependabot.yml
  • .github/workflows/benchmarks.yml
  • .github/workflows/cargo-audit-pr.yml
  • .github/workflows/ci.yml
  • .github/workflows/coverage.yml
  • .github/workflows/scorecard.yml
  • .github/workflows/security.yml
  • AGENTS.md
  • Cargo.toml
  • src/completion.rs
  • src/connection.rs
  • src/delimited.rs
  • src/logging.rs
  • src/tls/ca.rs
  • tests/additional_mysql_types.rs
  • tests/credential_leak_sweep.rs
  • tests/database_seeding_test.rs
  • tests/exit_code_3_auth.rs
  • tests/exit_code_5_io.rs
  • tests/integration/common.rs
  • tests/path_safety.rs
  • tests/stream_separation.rs
  • tests/test_execution_utilities.rs
  • tests/test_seed_data.rs
  • tests/test_support/cli.rs
  • tests/tls_classifier_rcgen.rs
  • tests/tls_integration.rs
  • tests/tls_variants_test.rs

Comment on lines +10 to +12
permissions:
contents: read

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🌐 Web query:

For codecov/codecov-action v6, what are the exact workflow permission and input requirements for OIDC uploads (use_oidc: true), and are there any private-repo prerequisites?

💡 Result:

For codecov/codecov-action v6 with use_oidc: true: Workflow permissions required: permissions: id-token: write Input requirements: - uses: codecov/codecov-action@v6 with: use_oidc: true Any token provided is ignored; the action fetches the OIDC token using core.getIDToken('https://codecov.io') (audience defaults to Codecov's URL). Private repo prerequisites: None explicitly stated beyond the above. OIDC works equivalently to tokens for private repos (authenticates uploads). For PRs from forks, OIDC fails (no id-token access, similar to secrets), falling back to tokenless (may require Codecov settings for private repos, but OIDC itself has no extra private-repo setup like audience config in sources).

Citations:


Migrate Codecov auth from long-lived token to OIDC.

The workflow currently grants only contents: read (lines 10-12) but depends on CODECOV_TOKEN secret (line 65). Adopt OIDC authentication instead: add id-token: write permission and use use_oidc: true in the codecov action. This eliminates the static CI secret, reducing credential exposure surface.

Proposed fix
 permissions:
     contents: read
+    id-token: write
@@
             - name: Upload to Codecov
               if: github.event_name != 'pull_request' ||
                   github.event.pull_request.head.repo.full_name == github.repository
               uses: codecov/codecov-action@3f20e214133d0983f9a10f3d63b0faf9241a3daa # v6.0.0
               with:
                   files: lcov.info
                   fail_ci_if_error: true
-                  token: ${{ secrets.CODECOV_TOKEN }}
+                  use_oidc: true
                   slug: EvilBit-Labs/gold_digger
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/coverage.yml around lines 10 - 12, The workflow currently
only sets permissions: contents: read but still relies on the CODECOV_TOKEN
secret; update the permissions block to include id-token: write (e.g., add
"id-token: write" alongside contents: read), change the Codecov action
invocation to use OIDC by setting use_oidc: true and remove the CODECOV_TOKEN
secret input (or guard its removal by switching to the OIDC flow in the codecov
action call), and ensure any references to CODECOV_TOKEN in the job steps are
deleted so the action authenticates via OIDC instead of the long‑lived secret.

Comment thread src/logging.rs
Comment on lines +56 to +84
/// Initialises the `tracing` subscriber with a plain-text stderr formatter
/// and a verbosity filter derived from CLI flags.
///
/// # Arguments
///
/// * `verbose` — the `-v` count from `cli.verbose` (u8).
/// * `quiet` — the `--quiet` flag from `cli.quiet`.
///
/// # Level mapping
///
/// | `quiet` | `verbose` | Level |
/// |---------|-----------|-------|
/// | `true` | *any* | error |
/// | `false` | 0 | warn |
/// | `false` | 1 | info |
/// | `false` | 2 | debug |
/// | `false` | ≥3 | trace |
///
/// The `RUST_LOG` env var, when set, overrides the CLI-derived filter.
/// This is intended for ad-hoc debugging; CI / production should drive
/// verbosity through the CLI flags.
///
/// # Idempotence
///
/// Safe to call at most once per process. Subsequent calls after the first
/// successful installation silently no-op (the second `set_global_default`
/// fails and we discard the error rather than panic — the subscriber is a
/// best-effort observability concern, not a correctness-critical boundary).
pub fn init_tracing(verbose: u8, quiet: bool) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion | 🟠 Major

Complete the required doc sections on the new public logging helpers.

init_tracing, stderr_supports_color, warn_banner, danger_banner, and make_progress still do not consistently provide the required Arguments, Returns, and Example sections. Please finish those while these APIs are still new.

As per coding guidelines, "src/**/*.rs: Doc comments (///) required for all public functions with Arguments, Returns, and Example sections`."

Also applies to: 129-206

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/logging.rs` around lines 56 - 84, Add the missing documentation sections
for the new public logging helpers: init_tracing, stderr_supports_color,
warn_banner, danger_banner, and make_progress. For each function add ///
Arguments describing parameters, /// Returns describing the return value (or
unit), and an /// Example showing minimal usage (copyable snippet) consistent
with existing crate style; keep wording concise and include any side-effects
(e.g., global subscriber installation in init_tracing or terminal output for
banners). Ensure the doc comments are placed immediately above each pub fn and
follow the project's three-section requirement (Arguments, Returns, Example).

Comment thread src/tls/ca.rs
Comment on lines +8 to +12
use super::error::TlsError;
use rustls_pki_types::{CertificateDer, pem::PemObject};
use std::fs::File;
use std::io::BufReader;
use std::path::{Path, PathBuf};
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Import ordering differs from coding guidelines.

The guidelines specify grouping imports by: std, external crates, then local modules (separated by newlines). Current order is local → external → std.

♻️ Suggested reordering
-use super::error::TlsError;
-use rustls_pki_types::{CertificateDer, pem::PemObject};
 use std::fs::File;
 use std::io::BufReader;
 use std::path::{Path, PathBuf};
+
+use rustls_pki_types::{CertificateDer, pem::PemObject};
+
+use super::error::TlsError;

As per coding guidelines: "Group imports by standard library, external crates, and local modules, separated by newlines."

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
use super::error::TlsError;
use rustls_pki_types::{CertificateDer, pem::PemObject};
use std::fs::File;
use std::io::BufReader;
use std::path::{Path, PathBuf};
use std::fs::File;
use std::io::BufReader;
use std::path::{Path, PathBuf};
use rustls_pki_types::{CertificateDer, pem::PemObject};
use super::error::TlsError;
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/tls/ca.rs` around lines 8 - 12, Imports in src/tls/ca.rs are misordered;
reorder them to follow the coding guideline: list std imports first (e.g.,
std::fs::File, std::io::BufReader, std::path::{Path, PathBuf}), then external
crate imports (e.g., rustls_pki_types::{CertificateDer, pem::PemObject}), and
finally local module imports (e.g., use super::error::TlsError), with a blank
line between each group to match the project's import ordering convention.

Comment on lines +163 to +166
.args([
"--tls-ca-file",
"/nonexistent/path/to/ca.pem",
"--db-url",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Use a tempdir-relative missing CA path here.

/nonexistent/path/to/ca.pem is host-dependent and awkward on Windows. A missing path under temp keeps the failure mode identical while making the test deterministic on every runner.

Suggested patch
     let temp = tempdir().expect("tempdir");
     let out = temp.path().join("out.json");
+    let missing_ca = temp.path().join("missing-ca.pem");
 
     let url = sentinel_url();
     let output = clean_cmd()
         .args([
             "--tls-ca-file",
-            "/nonexistent/path/to/ca.pem",
+            missing_ca.to_str().expect("utf-8 path"),
             "--db-url",
             &url,

As per coding guidelines, "tests/**: Review tests for: Cross-platform compatibility`."

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/credential_leak_sweep.rs` around lines 163 - 166, Replace the hardcoded
host-dependent missing CA path "/nonexistent/path/to/ca.pem" used in the
.args([...]) call in tests/credential_leak_sweep.rs with a deterministic
tempdir-relative path (e.g., create a TempDir via tempfile::tempdir() or use
std::env::temp_dir() and join a non-existent filename) and pass that joined path
to the "--tls-ca-file" argument so the test fails for a missing file
consistently across platforms (Windows/Linux/macOS).

Comment thread tests/exit_code_3_auth.rs
Comment on lines +28 to +51
let mut cmd = cargo::cargo_bin_cmd!("gold_digger");
cmd.env_remove("DATABASE_URL")
.env_remove("DATABASE_QUERY")
.env_remove("OUTPUT_FILE")
.env_remove("NO_COLOR")
.args([
"--db-url",
"mysql://baduser:badpass@127.0.0.1:1/db",
"--query",
"SELECT 1",
"--output",
// Point at a path inside a writable tempdir so the failure is
// unambiguously a DB-connection issue, not an output-write one.
output_path.to_str().expect("utf-8 path"),
])
.assert()
.code(3)
.stderr(
predicate::str::contains("connection")
.or(predicate::str::contains("connect"))
.or(predicate::str::contains("refused"))
.or(predicate::str::contains("MySQL"))
.or(predicate::str::contains("mysql")),
);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion | 🟠 Major

Route these auth tests through clean_cmd() instead of re-listing env removals.

These blocks bypass the shared helper and hand-maintain the scrub list. That already risks drift with the current binary: src/logging.rs::init_tracing also honors RUST_LOG, so parent-shell logging can still bleed into these runs. Building both commands via tests/test_support/cli.rs::clean_cmd() keeps exit-code coverage deterministic and future-proof.

As per coding guidelines, "tests/**/*.rs: Use .env_remove(\"DATABASE_URL\") etc. on Command to prevent user shell env leakage in integration tests`."

Also applies to: 64-78

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/exit_code_3_auth.rs` around lines 28 - 51, The test constructs the
Command by manually removing env vars (cargo::cargo_bin_cmd!("gold_digger") with
multiple .env_remove calls), which duplicates scrub logic and can miss vars like
RUST_LOG honored by src/logging.rs::init_tracing; replace this manual setup with
the shared helper tests/test_support/cli.rs::clean_cmd() so the test calls
clean_cmd(cargo::cargo_bin_cmd!("gold_digger")) (or the equivalent pattern used
elsewhere) and remove the explicit .env_remove calls to ensure all scrubbed envs
(including RUST_LOG) are consistently applied and keep exit-code assertions
unchanged.

Comment thread tests/test_support/cli.rs
Comment on lines +16 to +23
/// Environment variables that Clap reads via `#[arg(env = "...")]`. These
/// must be removed from any spawned binary so that user-shell exports do not
/// leak into integration tests (recorded as a recurring failure mode in
/// project memory). `NO_COLOR` is included for forward compatibility with
/// upcoming color/ANSI work.
pub const ENV_VARS_TO_REMOVE: &[&str] =
&["DATABASE_URL", "DATABASE_QUERY", "OUTPUT_FILE", "NO_COLOR"];

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Scrub RUST_LOG in the shared test command helper.

clean_cmd() is now the single source of truth for test-process hygiene, but src/logging.rs also honors parent RUST_LOG. A developer shell with RUST_LOG=trace can therefore inject extra stderr into helper-based tests and destabilize snapshots/assertions even when the Clap env vars are cleared.

Suggested patch
-pub const ENV_VARS_TO_REMOVE: &[&str] =
-    &["DATABASE_URL", "DATABASE_QUERY", "OUTPUT_FILE", "NO_COLOR"];
+pub const ENV_VARS_TO_REMOVE: &[&str] = &[
+    "DATABASE_URL",
+    "DATABASE_QUERY",
+    "OUTPUT_FILE",
+    "NO_COLOR",
+    "RUST_LOG",
+];

As per coding guidelines, "tests/**/*.rs: Use .env_remove(\"DATABASE_URL\") etc. on Command to prevent user shell env leakage in integration tests`."

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/test_support/cli.rs` around lines 16 - 23, The tests currently fail to
scrub RUST_LOG from spawned test processes; update the shared test helper so
clean_cmd() also removes "RUST_LOG" from the environment (i.e., add "RUST_LOG"
to the ENV_VARS_TO_REMOVE constant and ensure the code in clean_cmd() iterates
that list and calls .env_remove(...) for each entry) so parent-shell RUST_LOG
won't leak into test subprocesses or logging.rs behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL This PR changes 1000+ lines, ignoring generated files. type/chore Maintenance and infrastructure tasks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants