OpenSSL fixed a denial-of-service flaw that lets a remote, unauthenticated attacker exhaust a server’s memory with an 11-byte request during the TLS handshake, but the fix shipped without a CVE identifier, a security advisory, or even a changelog entry calling it a security fix. Okta’s security research team, which found and disclosed the issue under the name HollowByte, says that omission means most enterprise vulnerability scanners never flagged the patch as something to prioritize.
An 11-Byte Request That Locks Up Memory
The bug lives in how OpenSSL handles the start of a TLS handshake. Every handshake message carries a 4-byte header that declares how large the incoming message body will be. In versions before the fix, OpenSSL allocates a receive buffer sized to that declared length before a single byte of the actual message body arrives, and it does not validate the claim against anything. An attacker who sends a rogue header can force an allocation of up to roughly 131 KB per connection using a payload as small as 11 bytes, then simply never sends the rest of the data, leaving the worker thread holding that memory while it waits indefinitely.
Why the Damage Outlives the Connection
A single stalled connection is a nuisance, not an outage. The multiplier is glibc’s memory allocator, which does not hand small to medium freed allocations back to the operating system; it holds them for reuse instead. By opening waves of connections with randomized claimed sizes, an attacker prevents the allocator from ever reusing those freed chunks cleanly, so the heap fragments instead of settling. The server’s resident memory climbs, and it stays elevated even after the attacker disconnects, because the fragmented chunks never get consolidated back into memory the operating system can reclaim. In Okta’s own testing against unpatched NGINX instances, a 1 GB server was OOM-killed outright, and a 16 GB server had roughly a quarter of its system memory permanently locked in fragments after a single test run.
Who Is Exposed
Because OpenSSL underlies the TLS stack for a wide swath of common infrastructure, the exposure list is long: web servers including Apache and NGINX, language runtimes including Node.js, Python, Ruby, and PHP, and databases including MySQL and PostgreSQL, wherever they link an affected OpenSSL build. The fix landed in OpenSSL 4.0.1, 3.6.3, 3.5.7, 3.4.6, and 3.0.21, all dated June 9, 2026, delivered through incremental buffer growth changes rather than a rewrite of the handshake logic.
Detecting Exploitation Without a Signature
Because HollowByte has no CVE, it also has no vendor-issued detection rule to drop into a SIEM. Okta’s write-up points defenders toward behavior rather than a signature: watch for TLS-terminating processes whose resident set size climbs steadily across many short-lived, incomplete handshakes rather than in step with legitimate traffic growth, and correlate that with connection logs showing a high ratio of handshakes that open and then go silent before completing. A load balancer or reverse proxy that already logs handshake duration and bytes received per connection can surface this pattern with existing telemetry, no new tooling required, which matters for teams that cannot wait on a vendor detection signature that a silent patch will never produce.
What Makes This a Bigger Problem Than the Bug Itself
Most enterprise patch programs are built around CVE identifiers: a scanner checks installed versions against a CVE database, flags a gap, and a severity score decides how fast the gap gets closed. OpenSSL handled HollowByte as an internal hardening fix rather than a numbered security advisory, so it never entered that pipeline. A security team that patches strictly by CVE feed, rather than by tracking release notes for the libraries underneath its own stack, would have had no signal that this fix mattered, even though it closes a remotely triggerable, unauthenticated denial-of-service path in one of the most widely deployed cryptographic libraries in production use today.
What It Means for the Security Leader
Three things follow from this. First, a software bill of materials or dependency inventory that only tracks known-CVE exposure is not sufficient for foundational libraries like OpenSSL and glibc; teams need a process for reviewing release notes and hardening fixes for that narrow set of dependencies, independent of whether a CVE was ever assigned to the change. Second, patch cadence for that same narrow set of infrastructure libraries should not wait on a severity score that will never get calculated, because a silent fix has no score by definition. Third, if resident memory on internet-facing TLS terminators has been creeping upward without an obvious cause, it is worth checking that creep against the pattern Okta described rather than writing it off as ordinary load growth and restarting the process.
The Defender’s Checklist
Upgrade to OpenSSL 4.0.1, 3.6.3, 3.5.7, 3.4.6, or 3.0.21, or the equivalent patched build shipped by your Linux distribution, on every internet-facing service that terminates TLS. Audit your dependency inventory for the actual linked OpenSSL version rather than relying solely on a CVE scanner result to tell you whether you are affected. Add non-CVE changelog monitoring to your vulnerability management process for the handful of foundational libraries your stack depends on most. And if a public-facing server has shown unexplained, sustained memory growth, treat that as a signal worth investigating rather than a routine restart-and-move-on.
CyberTech has covered a related pattern before: vendors’ disclosed flaws being weaponized within hours of disclosure, compressing the time defenders have to react. HollowByte is close to the mirror image of that problem: a flaw serious enough to crash production servers, patched quietly enough that most defenders never got a signal to act on it at all.
Source: Okta Security
