HollowByte DDoS flaw causes OpenSSL server memory bloat with 11-byte payload

A vulnerability referred to as ‘HollowByte’ permits an unauthenticated attacker to trigger a denial of service (DoS) situation on an OpenSSL server with a malicious payload of simply 11 bytes.

The OpenSSL workforce mounted the vulnerability silently (no identifier was assigned) and backported the patch to an older launch.

As a result of OpenSSL software program is the underlying spine of safe Web communications, organizations ought to prioritize switching to a set model of the library.

Hole chunk particulars

In an advisory earlier this week, Okta’s Pink Crew defined how the HollowByte DoS vulnerability works and its impression in real-world situations.

The researchers clarify that in a TLS handshake, every message has a 4-byte header to declare the dimensions of the acquired message. Nonetheless, susceptible OpenSSL variations allocate the declared size earlier than receiving the payload and checking its dimension.

All TLS handshake messages start with a 4-byte handshake header adopted by a 3-byte size subject indicating the dimensions of the handshake knowledge.

With out validating the payload, the server trusts the packet’s request and allocates the desired reminiscence. “Employee threads are blocked, ready indefinitely for knowledge that by no means arrives,” Okta explains.

An unauthenticated attacker might set off HollowByte by opening a TLS connection and sending 11 bytes of malicious enter with a header that declares {that a} bigger message physique will observe.

The attacker repeats the identical course of throughout a number of connections, inflicting the server to allocate a considerable amount of reminiscence with a comparatively small quantity of knowledge being despatched.

Okta researchers level out that whereas OpenSSL frees buffers when a connection is damaged, the GNU C Library (glibc) handles reminiscence in a different way, “doing not instantly return small to medium-sized allocations to the working system, however retaining them for potential reuse.”

“The attacker prevents the allocator from reusing freed chunks by initiating a wave of connections with randomized request sizes,” Okta says.

“The heap turns into closely fragmented and the server’s resident set dimension (RSS) frequently will increase. Even after the attacker disconnects, the server stays completely bloated.”

The one technique to absolutely reclaim area is to restart the method.

Influence and fixes

The open supply OpenSSL library is constructed into fashionable software program initiatives reminiscent of NGINX and Apache internet servers, language runtimes (Node.js, Python, Ruby, PHP, and many others.), and databases (MySQL, PostgreSQL). Preinstalled on most Linux distributions for TLS encryption and certificates dealing with.

Okta testing on NGINX confirmed that whereas HollowByte can simply exhaust reminiscence in low-capacity environments, high-spec servers can lose as much as 25% of reminiscence whereas assault bandwidth stays beneath the safety warning threshold.

Though DoS flaws are usually not thought-about as critical as vulnerabilities that permit knowledge theft or code execution, they’ll trigger operational disruption and reputational harm.

The HollowByte DoS concern was mounted in OpenSSL 4.0.1 and backported to variations 3.6.3, 3.5.7, 3.4.6, and three.0.21 to disregard header claims and solely enhance the buffer as knowledge arrives.

Although it’s being addressed as a “hardening repair” quite than a safety vulnerability, Okta recommends “instantly improve your distribution’s OpenSSL packages.”