nowotarski.info

HTTP/2 CONTINUATION Flood

Posted on April 3, 2024 by Bartek Nowotarski
tl;dr: The CONTINUATION Flood is a class of vulnerabilities within numerous HTTP/2 protocol implementations. In many cases, it poses a more severe threat compared to the Rapid Reset: a single machine (and in certain instances, a mere single TCP connection or a handful of frames) has the potential to disrupt server availability, with consequences ranging from server crashes to substantial performance degradation. Remarkably, requests that constitute an attack are not visible in HTTP access logs.

What is the CONTINUATION Flood?

It is a new class of vulnerabilities in multiple implementations of HTTP/2 protocol. The root cause is an incorrect handling of HEADERS and multiple CONTINUATION frames which ultimately leads to Denial of Service.

The outcome depends on the implementation but ranges from an instant crash after sending a couple of HTTP/2 frames, Out Of Memory crash, to CPU exhaustion affecting server availability.

More technical details can be found in HTTP/2 CONTINUATION Flood: Technical Details post.

Affected projects

The maintainers of the following projects confirmed vulnerability (alphabetical order). Check respective security advisories for more information.

Show projects which inherited the vulnerabilityHide projects which inherited the vulnerability: projects using an affected server or importing an affected package. The list is not comprehensive.

Project Confirmed Affected Versions CVE ID Links
amphp/http 2024-03-11 >= 2.0.0 && <= 2.1.0, <= 1.7.2 CVE-2024-2653 Advisory
Apache HTTP Server (httpd) 2024-02-23 2.4.17-2.4.58 CVE-2024-27316 Advisory
Apache Tomcat 2024-01-25 <=11.0.0-M16, <=10.1.18, <=9.0.85, 8.5.0-8.5.98 CVE-2024-24549 Advisory*
↳ cPanel 2024-04-05 Forum
↳ Easy Apache 2024-04-09 Advisory
↳ IBM 2024-04-05 Advisory
↳ NetApp 2024-04-03 Advisory
Apache Traffic Server 2024-03-29 8.0.0-8.1.9, 9.0.0-9.2.3 CVE-2024-31309 Advisory
Envoy proxy (oghttp) 2024-02-27 1.29.0, 1.29.1 CVE-2024-27919 Advisory
Envoy proxy (nghttp2) 2024-02-27 <=1.29.2 CVE-2024-30255 Advisory
↳ Arista 2024-04-03 Advisory
Golang 2024-01-10 <=1.20, <=1.21.8, <=1.22.1 CVE-2023-45288** Advisory
↳ Arista 2024-04-03 Advisory
↳ Caddy 2024-04-04 Issue
↳ etcd 2024-04-04 Issue
↳ Gitea 2024-04-05 PR
↳ Google Cloud 2024-04-03 Advisory
↳ gRPC 2024-04-03 Commit
↳ Moby 2024-04-04 Commit
↳ Kubernetes 2024-04-03 Issue
↳ Traefik 2024-04-15 Advisory
h2 Rust crate 2024-03-04 Advisory
Mozilla Firefox 2024-01-17 <125 CVE-2024-3302 Advisory
Mozilla Thunderbird 2024-01-17 CVE-2024-3302
nghttp2 2024-03-08 <=1.60.0 CVE-2024-28182 Advisory
↳ Arista 2024-04-03 Advisory
Node.js 2024-01-15 <=18.20.0, <=20.12.0, <=21.7.1 CVE-2024-27983 Advisory
Tempesta FW 2024-03-16 0.7.0 CVE-2024-2758
█████████*** - - -
█████████ - - -
* Apache Tomcat issue was not directly related to CONTINUATION Flood but was revealed by an exploit for this vulnerability. Because of this Tomcat team decided to publish a new release before embargo end date.
** Golang team used an allocated-but-unused CVE ID left over from 2023, probably by mistake.
*** Projects affected that have not publicly confirmed it yet are currently hidden.

NOT affected: Nginx, Jetty, HAProxy, NetScaler, Varnish, Cloudflare.

Mitigation

Upgrade affected software to the latest version that contains the fix for this vulnerability. If no fix is available consider temporarily disabling HTTP/2 on the server.

Comparison to HTTP/2 Rapid Reset

Similarly to HTTP/2 Rapid Reset, CONTINUATION Flood can cause Denial of Service on an affected server. There are, however, a lot of differences which make CONTINUATION Flood a more severe attack technique:

  • Rapid Reset used a combination of HEADERS (with END_STREAM and END_HEADERS flags set) and RST_STREAM frames which means that standard mitigations like rate limiting could at least limit the damage. Also, the server admin would see a lot of inbound server requests and be alerted. During CONTINUATION Flood attack not a single request is made (no END_HEADERS flag)! Admins do not see any requests in the logs!
  • In many implementations, just one TCP connection was enough to crash the server (and in some cases with a very small amount of data sent) during the CONTINUATION Flood attack. On contrary, Rapid Reset was used in DDoS attacks (in most cases using a botnet was required for an attack to be successful).
  • Rapid Reset was exploited in the wild (check Cloudflare and Google posts). The CONTINUATION Flood was reported to CERT/CC so vendors were able to coordinate code fixes to prevent serious damage.

Comparison to older HTTP/2 vulnerabilities can be found in HTTP/2 CONTINUATION Flood: Technical Details post.

Is it a bug in HTTP/2 protocol?

It is an implementation bug in multiple HTTP/2 implementations. RFC9113 in section “10.5.1. Limits on Field Block Size” mentions multiple security issues that may arise if CONTINUATION frames are not handled correctly. At the same time, it does not mention a specific case in which CONTINUATION frames are sent without the final END_HEADERS flag which can have repercussions on affected servers.

Timeline

  • 2024-01-05: Issue discovered by Bartek Nowotarski.
  • 2024-01-08: Multiple reports sent to vendors.
  • 2024-01: Confirmations from Golang, Node.js, Apache Tomcat teams.
  • 2024-01-25: Report sent to CERT/CC.
  • 2024-02-14: Invitation to participate in vulnerability coordination from CERT/CC. Multiple vendors join.
  • 2024-02: Confirmations from Apache httpd, Envoy teams.
  • 2024-03: Confirmations from Apache Traffic Server, nghttp2, amphp, Tempesta teams.
  • 2024-04-03: Embargo on vulnerability information ends.
  • 2024-04-04: National CERTs/CSIRTs (Spain, Romania, Italy, Japan, Belgium, Singapore, France) warn about the vulnerability.
  • 2024-04-08: CISA.gov includes CONTINUATION Flood in the weekly Vulnerability Summary.
  • 2024-04-16: The vulnerability mentioned in DDoS threat report for 2024 Q1 by Cloudflare.
  • 2024-04-17: Mozilla Firefox vulnerable to HTTP/2 CONTINUATION Flood.

Technical Details

More technical details can be found in HTTP/2 CONTINUATION Flood: Technical Details post.

In the news