Skip to content

Default provider can crash with OpenSSL 3.5+ but without ML-KEM support #31

Description

@xnox

OpenSSL 3.5+ at runtime can be paired with providers that do not offer ML-KEM support. For example:

  • default provider when openssl compiled with no-ml-kem
  • validated FIPS providers without ml-kem capability (as in all current vintage of all validated FIPS providers)
  • Microsoft SymCrypt provider
  • Wolfcrypt providers
  • and possibly others

In such cases, even without the prefer-post-quantum feature, and with fips feature the connectivity can fail as follows:

$ RUST_BACKTRACE=full /work/target/release/rustls-openssl-client

thread 'main' (10) panicked at src/main.rs:49:10:
write HTTP request: Custom { kind: InvalidData, error: General("OpenSSL keygen error: error:0308010C:digital envelope routines:inner_evp_generic_fetch:unsupported:crypto/evp/evp_fetch.c:376:Global default library context, Algorithm (X25519MLKEM768 : 0), Properties (<null>)") }
stack backtrace:
   0:     0x563f6ca03722 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::hdadc7c28d26d7ce3
   1:     0x563f6ca130ea - core::fmt::write::h469843d235cc4241
   2:     0x563f6c9df0c6 - std::io::Write::write_fmt::h384251ddb7f1467c
   3:     0x563f6c9e7c26 - std::panicking::default_hook::{{closure}}::hd9d92b9a46805102
   4:     0x563f6c9e7a86 - std::panicking::default_hook::hedb2be0819bbe276
   5:     0x563f6c9e7e6b - std::panicking::panic_with_hook::h0fc23abecf35c6bd
   6:     0x563f6c9e7ce8 - std::panicking::panic_handler::{{closure}}::ha111c1a3ccf85d19
   7:     0x563f6c9e5fc9 - std::sys::backtrace::__rust_end_short_backtrace::h41d63216e1ba5fa4
   8:     0x563f6c9d407d - __rustc[16f1505adc47261a]::rust_begin_unwind
   9:     0x563f6ca1756c - core::panicking::panic_fmt::hc13412975f563dde
  10:     0x563f6ca16ee2 - core::result::unwrap_failed::hd8abf8e6faab58b4
  11:     0x563f6c938965 - rustls_openssl_client::main::h8c3590d212edfe3a
  12:     0x563f6c93def3 - std::sys::backtrace::__rust_begin_short_backtrace::h00d81e23a834aa6e
  13:     0x563f6c93dee9 - std::rt::lang_start::{{closure}}::h1c6eccf025934c51
  14:     0x563f6c9e0876 - std::rt::lang_start_internal::hc68d929ebd5f7eea
  15:     0x563f6c93a355 - main
  16:     0x7fdcd67d1215 - __libc_start_call_main
  17:     0x7fdcd67d1338 - __libc_start_main@GLIBC_2.2.5
  18:     0x563f6c930ad5 - _start
  19:                0x0 - <unknown>

Instead of trying to gate the available KEM based on the version of OpenSSL (libcrypto.so) at compile time, it is best to check algorithm support as a runtime setting instead. Specifically, try to fetch the KEM and ciphers at init, and only use them if the algorithms are available.

This would also avoid the need to use custom provider, or even use fips feature in most cases - making binaries more universal (i.e. it will use X25519 if it is available at runtime, not whether or not it was available at compile time).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions