Start REVIEW_1 workspace and credential security foundation#80
Merged
Conversation
Agent-Logs-Url: https://github.com/bluehexagons/infra_tools/sessions/416e3f9f-eec6-4d45-b7bc-14312e4b3ba4 Co-authored-by: bluehexagons <6945365+bluehexagons@users.noreply.github.com>
Agent-Logs-Url: https://github.com/bluehexagons/infra_tools/sessions/416e3f9f-eec6-4d45-b7bc-14312e4b3ba4 Co-authored-by: bluehexagons <6945365+bluehexagons@users.noreply.github.com>
Agent-Logs-Url: https://github.com/bluehexagons/infra_tools/sessions/416e3f9f-eec6-4d45-b7bc-14312e4b3ba4 Co-authored-by: bluehexagons <6945365+bluehexagons@users.noreply.github.com>
Agent-Logs-Url: https://github.com/bluehexagons/infra_tools/sessions/416e3f9f-eec6-4d45-b7bc-14312e4b3ba4 Co-authored-by: bluehexagons <6945365+bluehexagons@users.noreply.github.com>
Agent-Logs-Url: https://github.com/bluehexagons/infra_tools/sessions/416e3f9f-eec6-4d45-b7bc-14312e4b3ba4 Co-authored-by: bluehexagons <6945365+bluehexagons@users.noreply.github.com>
Copilot created this pull request from a session on behalf of
bluehexagons
April 4, 2026 13:59
View session
Route system type discovery and default metadata through a built-in plugin registry, add conflict tests, and update REVIEW_1 progress notes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Resolve system steps through lazy plugin-owned builders, relocate shared step catalogs out of plugin discovery, and update REVIEW_1 progress notes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove direct shell=True usage from browser and Node helper paths, prefer argv execution for plain remote_utils commands, and add regression coverage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add shared SSH/SCP/rsync command builders, route setup/recall/deploy flows through them, and update REVIEW_1 progress notes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- lib/proxmox_hosts.py: workspace registry of Proxmox hosts (proxmox_hosts.json, 0600) with add/remove/find helpers. - lib/proxmox_manage.py: container ops over SSH (list, status, ip, start, stop, destroy, health_check) reusing _ssh_opts/_ssh_run from lib/proxmox_node.py. - lib/proxmox_shell.py: interactive REPL with injectable IO and a destroy-confirmation callback for testability. - lib/proxmox_cli.py + infra_tools.py: new 'proxmox' subcommand wiring shell, hosts, add, remove, ls, status, start, stop, destroy (-y), and health subcommands. - tests/expensive_support.py: @Expensive decorator gated on INFRA_TOOLS_RUN_EXPENSIVE=1 so slow/live tests stay opt-in. - tests/test_proxmox_*.py: 85 new tests; placeholder live test stays skipped unless the env var is set. - README.md: document the new proxmox subcommand and the expensive test gate. Future work (not yet wired): a 'proxmox create' subcommand wrapping lib.proxmox_node.provision_container, and modify/redeploy flows. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- tests/expensive_support.py: introduce expensive-test *categories* (live_proxmox, network, slow). Each maps to INFRA_TOOLS_RUN_<NAME>; INFRA_TOOLS_RUN_EXPENSIVE=1 still enables everything. The @Expensive(category, reason) decorator builds a useful skip message pointing at the right env var. - run_tests.py: rewrite with argparse. Adds --expensive CATEGORY (repeatable, 'all' supported), --list-categories, --list-tests, arbitrary positional selectors (file stems or dotted module / test ids). Default suite stays fast and concise. - tests/test_proxmox_live.py: replace the placeholder with a real, manually-run lifecycle test that creates an LXC via 'pct create', exercises list/status/start/stop/health, and destroys it. Gated on the live_proxmox category and SkipTest if PROXMOX_TEST_HOST / PROXMOX_TEST_TEMPLATE are absent. tearDown always destroys the container so failures don't leave debris. - tests/test_expensive_support.py: cover the new category gating and decorator behaviour. - README.md: document the test runner and expensive-test categories. Default suite: 1153 tests pass, 1 skipped (the live Proxmox test). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- run_tests.py: add named suites (smoke, proxmox, security, integration, all), --list-suites, --durations N, and --check-prereqs for requested expensive categories. - tests/test_proxmox_live.py: expose check_live_proxmox_prereqs() for the runner, validate required env vars / VMID / SSH key path, and verify the target host has pct before destructive live runs. - tests/test_run_tests.py: cover suite discovery, prereq reporting, and duration output. - README.md: document suite runs, duration reporting, and prereq checks. Validation: - ./run_tests.py --suite smoke --durations 5 - ./run_tests.py --suite proxmox - ./run_tests.py Default suite: 1159 tests pass, 1 skipped (live Proxmox). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- lib/proxmox_manage.py: add native Proxmox webhook notification setup via pvesh. The installer upserts a webhook endpoint and matcher, uses Proxmox's expected base64-encoded body/header property strings, and can trigger Proxmox's native target test endpoint. - lib/proxmox_node.py: allow callers to provide a redacted log command so webhook URLs with query tokens are not printed while the actual remote command still receives the full URL. - lib/proxmox_cli.py: add 'proxmox notifications install-webhook' and 'test-webhook' subcommands with endpoint/matcher names, severity filtering, dry-run, and optional test notification support. - tests: cover pvesh command generation, validation, URL redaction, CLI dispatch, dry-run, and test notification commands. - README.md: document native webhook notification setup. Validation: - ./run_tests.py test_proxmox_manage test_proxmox_cli --durations 5 - ./run_tests.py --suite proxmox - ./run_tests.py Default suite: 1170 tests pass, 1 skipped (live Proxmox). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add --antistatic-db DOMAIN[:PORT] setup flag and SetupConfig propagation through remote args and setup command rendering. - Extend server plugin Antistatic steps to optionally deploy antistatic-db alongside antistatic-server. - Add release-only antistatic-db installation flow expecting future GitHub assets named antistatic-db-linux-amd64 / antistatic-db-linux-arm64 from bluehexagons/antistatic-db releases. - Add antistatic-db systemd unit generation with loopback binding, nginx proxy trust, StateDirectory-backed SQLite storage under /var/lib/antistatic-db, restart policy, and hardening matching antistatic-server. - Include antistatic-db.service in infra_tools systemd cleanup patterns. - Update README, command-line docs, AGENTS notes, and tests for parsing, release discovery/download, service generation, and config wiring. Validation: - ./run_tests.py test_antistatic_steps test_config --durations 5 - ./run_tests.py Default suite: 1185 tests pass, 1 skipped (live Proxmox). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add a shared cleanup command timeout so maintenance commands cannot hang indefinitely. - Apply the timeout to optional cleanup commands and nvm cleanup commands, reporting timeouts as structured failures for logs and notifications. - Add weekly apt-get autoremove to cleanup-maintenance, in addition to autoclean and clean, to reclaim unused packages outside auto-update runs. - Update deployment safety docs and README to reflect autoremove and bounded cleanup command runtime. - Extend cleanup-maintenance tests for timeout handling, nvm timeout reporting, and autoremove command wiring. Validation: - python3 -m py_compile common/service_tools/cleanup_maintenance.py lib/maintenance_defaults.py tests/service_tools/test_cleanup_maintenance.py - ./run_tests.py service_tools/test_cleanup_maintenance --durations 8 - ./run_tests.py test_security_steps service_tools/test_cleanup_maintenance --durations 10 - ./run_tests.py Default suite: 1188 tests pass, 1 skipped (live Proxmox). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add infra_tools-owned temp artifact prefixes and a 7-day retention constant to maintenance defaults. - Extend cleanup-maintenance to remove stale /tmp artifacts left by interrupted setup, deploy, recall, Proxmox public-key upload, and Antistatic release-download runs. - Keep cleanup conservative: only known infra_tools prefixes are considered, fresh files are preserved, and removal failures are logged and included in existing failure notifications. - Update cleanup-maintenance tests for stale file/dir removal, preservation of fresh/unrelated paths, and removal failures. - Update README and deployment safety docs to document stale infra_tools artifact cleanup. Validation: - python3 -m py_compile common/service_tools/cleanup_maintenance.py lib/maintenance_defaults.py tests/service_tools/test_cleanup_maintenance.py - ./run_tests.py service_tools/test_cleanup_maintenance --durations 10 - ./run_tests.py test_security_steps service_tools/test_cleanup_maintenance --durations 10 - ./run_tests.py Default suite: 1190 tests pass, 1 skipped (live Proxmox). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep the mocked rmtree failure scoped to the cleanup helper call so Python 3.14 TemporaryDirectory teardown can use the real shutil.rmtree in CI. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ruby is already apt-managed (no rbenv), so the only remaining cleanup gap was the bundler/gem build temp directories left behind in /var/tmp by interrupted Rails deploys (lib/deployment.py forces TMPDIR=/var/tmp for 'bundle install' so they don't fill /tmp). - Add 'bundler' to INFRA_TMP_PREFIXES so bundlerYYYYMMDD-PID-RANDOM dirs older than the stale-temp retention are removed. - Add INFRA_TMP_DIRS so the stale-temp pass scans both /tmp and /var/tmp. - Loop cleanup_stale_infra_tmp_artifacts over INFRA_TMP_DIRS in the main cleanup_maintenance run. - Tighten install_ruby's bundler install with --no-document. - Update tests for the new prefix, the multi-dir loop, and the gem flag. - Note the broader scope in docs/DEPLOYMENT_SAFETY.md. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds a small infra_tools launcher script to bootstrap so users can run the CLI from any directory without invoking python3 explicitly: - Bootstrap (now also reachable as 'self-setup') drops a launcher into /usr/local/bin/infra_tools when run as root. - python-tools drops a launcher into ~/.local/bin/infra_tools and now registers shell completion for both infra_tools and infra_tools.py. - Bootstrap forwards the resolved script path through to python-tools. - README documents the new self-setup alias and launcher locations. Closes part of #79. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Implements the top-level 'shell' subcommand providing a REPL for managing saved configurations and accessing the proxmox shell: - New lib/interactive_shell.py with InteractiveShell REPL class. - Supports list/info/cmd/deploy/rm for saved configurations. - Supports recall, reconstruct, and proxmox subshell access. - workspace command to view/change active workspace. - Driver-agnostic design with testable input/output functions. - Comprehensive test coverage in tests/test_interactive_shell.py. - Updated COMMAND_LINE.md documentation. Closes #27 (interactive CLI tool). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add IDEAS.md capturing remaining work, open issues, and enhancement ideas. This file lists next steps for container lifecycle, interactive shell improvements, /tmp cleanup, testing, and docs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Implements #81: container config display, reconfigure (pct set), CPU/memory modification, and disk resize via pct resize. Adds proxmox CLI subcommands (config, reconfigure, modify, resize-disk) and matching ProxmoxShell commands. Adds readline persistent history to the interactive shell for #27. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds /tmp and /var/tmp usage logging to each cleanup_maintenance run so usage trends appear in structured logs. Installs /etc/tmpfiles.d/infra_tools.conf during bootstrap (requires root) to age out known infra_tools temp prefixes via systemd-tmpfiles-clean as a safety-net complement to cleanup_maintenance. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
list --json outputs a JSON array for scripting; info --compact prints one line per configuration. Both flags work in the interactive shell and on the CLI. InteractiveShell now reads ~/.infra_toolsrc at startup, dispatching each non-comment line as a shell command. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Covers new Proxmox lifecycle commands (config/modify/reconfigure/resize-disk), interactive shell features (--json/--compact, ~/.infra_toolsrc, shell history), bootstrap tmpfiles.d installation, and the proxmox management reference in COMMAND_LINE.md. Cleans up stale approach notes from the plans doc. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
infra_tools.py credentialscommands plus runtime credential resolution for Samba and SMB mountsplugins/package in remote setup artifact bundling so plugin-based system-type discovery works on live remote runsinfra_tools.pyentry point instead of removed legacy wrapper scriptsdocs/ideas/REVIEW_1.mdas a current implementation/status document for this PRshell=TrueValidation
python3 -m unittest discover -s tests✅ (977 tests)parallel_validation✅ Code Review + ✅ CodeQL on the final pass