BlackTree Security · Infrastructure · Automation · AI

A Stalled DTLS Handshake Could Make OpenSSL Send Heap Memory in Plaintext

OpenSSL has patched a high-severity flaw that can make an application send unintended heap memory to the other side of a DTLS connection as plaintext handshake data. If the read crosses into unmapped memory, the same defect can crash the process instead.

The OpenSSL DTLS flaw, CVE-2026-84782, is serious, but it is not a weakness in every HTTPS connection. It requires Datagram Transport Layer Security, a handshake write that stalls partway through, and a retransmission while that write remains suspended. Ordinary TLS over TCP does not use the affected path.

That distinction should guide prioritisation, not delay it. Every OpenSSL branch analysed in the 29 September advisory is affected, including premium-supported legacy releases.

The OpenSSL DTLS flaw starts with a stalled retransmission

DTLS carries TLS-style security over datagrams. OpenSSL may split a handshake message into fragments, and a write can return WANT_WRITE when the underlying transport temporarily cannot accept more data.

According to OpenSSL, a retransmission timer can fire while that write remains suspended. The retransmission logic reused the same buffer and position tracking without resetting the read position to the start of the message being resent. The result can be a wrongly labelled handshake message containing leftover bytes from another message, followed by an out-of-bounds read beyond the allocated buffer.

The peer may receive those unintended heap bytes in plaintext. OpenSSL does not say an attacker can choose the exact memory returned, and the public advisory does not report confirmed malicious exploitation. The alternative outcome is a crash and denial of service.

The fix resets the retransmission position and defers retransmission while another handshake write is suspended.

OpenSSL 4.0 has a second concurrency problem

The same release fixes CVE-2026-84783, a Moderate use-after-free limited to OpenSSL 4.0.

OpenSSL caches decoded X.509 extension data the first time an application needs it. If several threads build certificate chains to the same trusted CA at the same time, one thread can replace and free cached values while another thread is still using them.

A remote, unauthenticated peer may be able to crash a multi-threaded TLS client, or a TLS server that requests client certificates, during those first concurrent chain builds. Peer certificates are decoded separately for each connection and are not the shared objects at issue. The exposure concerns shared trusted CA certificates.

The bulletin contains 14 CVEs with very different reach

The number of CVEs is not the risk model. As BlackTree noted in its Firefox 155 analysis, the important questions are which boundary fails, what conditions make the code reachable and what the attacker gains.

CVE Severity What can happen Affected OpenSSL branches
CVE-2026-84782 High A suspended DTLS handshake write followed by retransmission can disclose heap bytes to the peer or crash the process. 4.0, 3.6, 3.5, 3.4, 3.0, 1.1.1, 1.0.2
CVE-2026-84783 Moderate Concurrent first use of a shared trusted CA certificate can trigger an X.509 cache use-after-free and crash. 4.0
CVE-2026-35189 Low A crafted certificate can turn roughly 100 KiB of certificate data into several hundred MiB of memory pressure. 4.0, 3.6, 3.5, 3.4, 3.0, 1.1.1, 1.0.2
CVE-2026-35191 Low A QUIC server with address validation disabled can overcount amplification credit and help amplify denial-of-service traffic. 4.0, 3.6, 3.5
CVE-2026-42772 Low A remote QUIC peer that completes the handshake can force quadratic stream-fragment reassembly work and consume CPU. 4.0, 3.6, 3.5, 3.4
CVE-2026-54872 Low Timing leakage in generic non-NIST elliptic curves may enable private-key recovery after many signing measurements. 4.0, 3.6, 3.5, 3.4, 3.0, 1.1.1, 1.0.2
CVE-2026-54873 Low A QUIC peer can make stream fragments retain packet-buffer memory longer than necessary. 4.0, 3.6, 3.5, 3.4
CVE-2026-54875 Low SM2 private-key operations leak timing or cache information on ARM64 and RISC-V. 4.0, 3.6, 3.5, 3.4
CVE-2026-72897 Low A specialised mid-handshake SSL_CTX switch can cause an out-of-bounds read or fixed-value write and denial of service. 4.0, 3.6, 3.5, 3.4
CVE-2026-75804 Low Missing connection-level QUIC stream flow control can let a peer drive memory use towards about 100 MB per connection. 4.0, 3.6, 3.5, 3.4
CVE-2026-75805 Low A malicious CMP server, or a suitably positioned attacker with the protection secret, can crash a CSR-based revocation client. 4.0, 3.6, 3.5, 3.4, 3.0
CVE-2026-75806 Low One unauthenticated undersized DTLS 1.2 AEAD datagram can terminate an existing association. 4.0, 3.6, 3.5, 3.4, 3.0, 1.1.1
CVE-2026-77696 Low SM2 signature generation leaks timing information that may support private-key recovery after many measurements. 4.0, 3.6, 3.5, 3.4, 3.0, 1.1.1
CVE-2026-84784 Low A QUIC peer can create a large RETIRE_CONNECTION_ID backlog, potentially consuming about 400 MB. 4.0, 3.6, 3.5, 3.4

The fixed version depends on the branch

Branch Fixed version Availability
OpenSSL 4.0 4.0.3 Public supported release
OpenSSL 3.6 3.6.5 Public supported release
OpenSSL 3.5 3.5.9 Public supported release
OpenSSL 3.4 3.4.8 Public supported release
OpenSSL 3.0 3.0.23 Premium support customers only
OpenSSL 1.1.1 1.1.1zj Premium support customers only
OpenSSL 1.0.2 1.0.2zs Premium support customers only

OpenSSL says branches 3.1, 3.2 and 3.3 are out of support and were not analysed. That does not mean they are unaffected. Operators should move unsupported deployments to a supported branch through the product or operating-system vendor rather than extrapolate safety from an absent row.

For CVE-2026-84782, OpenSSL says the affected DTLS code sits outside the FIPS module boundary. The advisory marks FIPS impact as no for all fourteen issues and says the listed FIPS modules are unaffected, although the reason varies by issue. For example, SM2 is not a FIPS algorithm, while the affected generic elliptic-curve path is not used by the approved NIST curves in the FIPS provider. That is not the same as application immunity. An application can use a validated cryptographic module while its DTLS, QUIC, X.509, CMP or surrounding protocol code remains exposed.

What defenders should do now

  • Find the OpenSSL that applications actually use. Check running processes, containers, appliances, firmware, language runtimes and statically linked binaries. A patched shared library on disk does not update a product that embeds its own copy.
  • Prioritise reachable features. Start with internet-reachable DTLS services and OpenSSL 4.0 applications that build certificate chains concurrently. Then identify QUIC, SM2, CMP and custom provider or SSL_CTX switching paths.
  • Use product and distribution updates where available. Downstream vendors may backport fixes without adopting OpenSSL’s version number. Follow their package guidance and verify the corrected build is loaded after restart or redeployment.
  • Refresh immutable assets. Rebuild container images, golden images, firmware bundles and application packages that contain the library. Record the deployed artefact, not only the approved update ticket.
  • Review exposed DTLS endpoints. Look for unusual handshake retransmissions, crashes and restarts. A lack of obvious logs does not prove that no heap data was returned, and crash dumps may themselves contain sensitive material.
  • Retest after remediation. Confirm the running process, protocol path and externally reachable service changed. Do not close the issue solely because a package manager reports success.

OpenSSL’s advisory does not report exploitation, and BlackTree found no credible public proof of concept during its 30 September review. The patch should still be treated as urgent wherever the High DTLS path is reachable. A narrow prerequisite can reduce the number of exposed systems while increasing the importance of identifying them correctly.

Sources

Leave a Reply

Your email address will not be published. Required fields are marked *