Skip to content

fix: install rustls ring CryptoProvider before TLS clients init#3

Merged
ZhiXiao-Lin merged 1 commit into
mainfrom
fix/rustls-crypto-provider
May 16, 2026
Merged

fix: install rustls ring CryptoProvider before TLS clients init#3
ZhiXiao-Lin merged 1 commit into
mainfrom
fix/rustls-crypto-provider

Conversation

@ZhiXiao-Lin
Copy link
Copy Markdown
Contributor

Summary

Fixes the tokio-rt-worker panic on startup that crashed pods running with the kube and redis features enabled (i.e. the v1.0.1 OCI image used as a Kubernetes Ingress Controller). This was deterministically reproducible against any cluster — every pod hit it as soon as the Ingress watcher opened its first TLS connection to the apiserver.

Root cause

rustls 0.23 requires an explicit process-level CryptoProvider when both aws-lc-rs and ring are present in the dep graph. With features kube + redis enabled, both backends get pulled in transitively, and rustls refuses to auto-pick:

thread 'tokio-rt-worker' (7) panicked at rustls-0.23.40/src/crypto/mod.rs:249:14:
Could not automatically determine the process-level CryptoProvider from Rustls crate features.

Logs from the failing pod show the panic landing right after Kubernetes Ingress watcher started — exactly when kube-rs constructs its first rustls client to talk to apiserver.

Fix

Install rustls::crypto::ring::default_provider() at the top of main(), before any TLS-using component (kube-rs, reqwest, server entrypoints) is constructed. ring is already what proxy/tls.rs:151 uses for server-side TLS, so this matches existing intent.

Verification

  • cargo build --features kube,redis --release
  • cargo test --features kube,redis --lib → 1027 passed, 0 failed ✅
  • Real-world repro: cluster at 10.12.111.133 had the v1.0.1 image in CrashLoopBackOff (7 restarts in 13 min); panic stack matches above.

Versioning

  • Cargo.toml: 1.0.1 → 1.0.2
  • deploy/helm/a3s-gateway/Chart.yaml: 1.0.0 → 1.0.2 (was lagging)
  • CHANGELOG.md: 1.0.2 entry added

With both `kube` and `redis` features pulling rustls 0.23 alongside
`aws-lc-rs` and `ring` transitively, rustls refuses to auto-pick a
process-level CryptoProvider and panics on first TLS use. This crashed
the Kubernetes Ingress watcher's apiserver connection on startup:

    Could not automatically determine the process-level CryptoProvider
    from Rustls crate features.

Install `rustls::crypto::ring::default_provider()` at the top of `main()`
before any TLS-using component (kube-rs, reqwest, server entrypoints) is
constructed. `ring` is already what `proxy/tls.rs` uses for server-side
TLS, so this matches existing intent.

Bumps a3s-gateway 1.0.1 → 1.0.2 and Helm chart 1.0.0 → 1.0.2.
@ZhiXiao-Lin
Copy link
Copy Markdown
Contributor Author

CI status note

The MSRV Check job is failing on a pre-existing unused variable: \s`warning ata3s-acl/src/parser.rs:160, surfaced because the gateway's CI sets RUSTFLAGS=-D warningsglobally, which propagates to the path-depa3s-acl` it pulls in.

This failure is not introduced by this PR — every commit on main since May 12 (including 3fdad48 v1.0.1) has the same MSRV failure. Verified by inspecting the latest CI runs on main.

Other PR jobs are healthy:

  • ✅ Lint (cargo clippy -p a3s-gateway --all-features -- -D warnings)
  • ✅ Test (cargo test -p a3s-gateway --lib → 1027 passed)
  • ✅ Documentation
  • ✅ Release Build (cross-compile)
  • ✅ Benchmark Compile

Notably, release.yml does NOT set RUSTFLAGS=-D warnings and gates on gateway-scoped clippy only, so tagging v1.0.2 will trigger image publish without being blocked by the upstream a3s-acl warning.

The acl warning belongs in a separate PR against A3S-Lab/ACL.

@ZhiXiao-Lin ZhiXiao-Lin merged commit bf4029a into main May 16, 2026
5 of 6 checks passed
@ZhiXiao-Lin ZhiXiao-Lin deleted the fix/rustls-crypto-provider branch May 16, 2026 03:20
ZhiXiao-Lin added a commit to A3S-Lab/a3s that referenced this pull request May 16, 2026
Picks up A3S-Lab/Gateway#3: install rustls::crypto::ring as the
process-level CryptoProvider before any TLS-using component constructs.
Resolves the `tokio-rt-worker` panic that crashed v1.0.1 pods on every
cluster as soon as the kube-rs Ingress watcher opened its first TLS
connection to the apiserver.

3fdad48 → bf4029a

Co-authored-by: Roy Lin <roylin@a3s.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant