Watch-Only Mode for ConfigMap Monitoring (And updating test framework)

[matthsmi]

PR: Watch-Only Mode for ConfigMap Monitoring

Summary

This PR introduces a watch-only mode for butler that enables monitoring configuration files (such as Kubernetes ConfigMaps) without writing to disk. Instead of copying files, butler hashes the source files and triggers reloads when changes are detected. This is ideal for read-only filesystem scenarios.

Additionally, this PR includes significant test infrastructure improvements, CI/CD enhancements, and build system cleanup.

New Features

Watch-Only Mode (watch-only = "true")

A new per-manager configuration option that changes butler's behavior:

• Hashes source files from repo-path using SHA-256
• Compares hashes to previous run (stored in memory)
• Triggers the configured reloader if hashes differ
• Never writes to dest-path (which becomes optional)

Use Cases

• Kubernetes ConfigMap monitoring where files are mounted read-only
• GitOps scenarios where configs are managed by Helm/ArgoCD
• Reducing disk I/O overhead when files are already in place

Example Configuration

[jenkins]
  repos = ["jcasc-local"]
  watch-only = "true"
  skip-butler-header = "true"
  primary-config-name = "jcasc-config.yaml"

  [jenkins.jcasc-local]
    method = "file"
    repo-path = "/usr/share/jenkins/init.jcasc.d"
    primary-config = ["config.yaml", "credentials.yaml"]

  [jenkins.reloader]
    method = "https"
    [jenkins.reloader.https]
      host = "localhost"
      port = "8080"
      uri = "/reload-configuration-as-code/?casc-reload-token=mytoken"
      method = "post"

Key Behaviors

Scenario	Behavior
First run	Always triggers reload (no previous hashes)
Container restart	Triggers reload (in-memory hashes lost)
File unchanged	No reload triggered
File changed	Reload triggered, hash updated

Files Changed

Core Implementation

File	Changes
internal/config/manager.go	Added WatchOnly and FileHashes fields to Manager struct
internal/config/helpers.go	Added ComputeFileHash(), ComputeDataHash(), CompareHashOnly() functions
internal/config/chan.go	Added ComparePrimaryConfigHashes() and CompareAdditionalConfigHashes() methods
internal/config/handler.go	Updated RunCMHandler() to use hash comparison in watch-only mode; updated CheckPaths() to skip directory operations
contrib/butler.toml.sample	Added documentation for watch-only option
README.md	Added comprehensive documentation for watch-only mode

New Test Files

File	Coverage
internal/config/status_test.go	Status file read/write operations
internal/config/objects_test.go	ValidateOpts builder, RepoFileEvent setters
internal/config/chan_test.go	ConfigChanEvent operations, temp file handling
internal/config/manager_test.go	Manager methods, path cleanup logic
internal/reloaders/reloaders_test.go	ReloaderError, GenericReloader
internal/reloaders/http_test.go	HTTP reloader functionality
internal/alog/alog_test.go	Apache logging handler
internal/methods/methods_test.go	Method factory for various schemes

Updated Test Files

File	Changes
internal/config/helpers_test.go	Added tests for hash functions and CompareHashOnly()
internal/config/config_test.go	Added tests for watch-only mode parsing

CI/CD & Build System

File	Changes
.github/workflows/pr.yml	Split into parallel test-unit and test-accept jobs
.github/workflows/release.yml	Split into parallel test-unit and test-accept jobs
docker/Dockerfile	Added tests for internal/environment and internal/alog; removed references to non-existent files
Makefile	make test now runs only unit tests; added make test-all for both
files/certs/generate_certs.sh	New script to regenerate test certificates
files/certs/*	Regenerated with proper SANs for Go 1.15+ compatibility

Removed Files

File	Reason
files/Dockerfile-build	Consolidated into docker/Dockerfile
files/Dockerfile-testaccept	Consolidated into docker/Dockerfile
files/Dockerfile-testunit	Consolidated into docker/Dockerfile

Test Plan

• Unit tests pass (make test-unit)
• Acceptance tests pass (make test-accept)
• All tests pass (make test-all)
• Watch-only mode correctly detects file changes via hash comparison
• Watch-only mode triggers reloader when hashes differ
• Watch-only mode does not write to dest-path
• dest-path is optional when watch-only = "true"
• First run always triggers reload in watch-only mode
• TLS certificates work with Go 1.15+ (SANs included)

Breaking Changes

None - this release is fully backward compatible. The watch-only option defaults to "false".

Labels

• release:minor - New feature addition