HTTP/2 Protocol Overview & Frame...

Introduction

HTTP/2 is the first major revision of the HTTP protocol since HTTP/1.1 (1999). It was standardized as RFC 7540 in 2015 and is now the default for most modern browsers and CDNs. The protocol introduces binary framing, multiplexed streams, header compression, and more robust flow control, all designed to reduce latency and improve throughput.

For security professionals, understanding HTTP/2 is crucial because many classic web-application attacks - such as request smuggling, cache poisoning, or denial-of-service - manifest differently under the new framing rules. Moreover, the binary nature of the protocol hides many parsing bugs that were previously visible in plain-text HTTP/1.1.

Real-world relevance: major platforms (Google, Cloudflare, Amazon) rely on HTTP/2 for performance. Attackers have already crafted exploits that abuse the SETTINGS and PRIORITY frames to bypass WAFs, while defenders need to know how to inspect frames with tools like nghttp2 or Wireshark.

Prerequisites

Basic HTTP fundamentals - methods, status codes, header syntax.
Understanding of TCP/IP and TLS basics - especially how TLS handshake precedes HTTP/2.
Familiarity with binary data inspection (e.g., hex editors, Wireshark).

Core Concepts

HTTP/2 is a binary protocol built on top of a single TCP connection (often TLS). The connection is divided into streams, each identified by a 31-bit stream identifier. Every piece of data - headers, payload, control information - is encapsulated in a frame. Frames are the atomic unit of communication and are always 9 bytes of header followed by a variable-length payload.

Key concepts:

Connection Preface: A client-initiated string (24 bytes) followed by an initial SETTINGS frame.
Multiplexing: Multiple streams can be interleaved on the same TCP connection without head-of-line blocking.
Flow Control: Both connection-level and stream-level windows limit how much data can be in flight.
Header Compression (HPACK): Reduces overhead by indexing frequently used header fields.
State Machines: Each endpoint maintains stream state (idle, open, half-closed, closed).

Understanding these fundamentals is the baseline for any security analysis of HTTP/2 traffic.

HTTP/2 connection preface and settings exchange

The very first bytes sent by a client are the literal string:

PRI <* HTTP/2.0

Followed immediately by a SETTINGS frame (type 0x04). The SETTINGS frame is used to negotiate parameters such as:

SETTINGS_MAX_CONCURRENT_STREAMS - maximum number of streams the peer may open.
SETTINGS_INITIAL_WINDOW_SIZE - default flow-control window for new streams.
SETTINGS_HEADER_TABLE_SIZE - HPACK dynamic table size.

Both sides may send additional SETTINGS frames at any time; each SETTINGS frame is acknowledged with a SETTINGS ACK (same frame with ACK flag set).

Example: Client sends preface and SETTINGS

# Using hyper-h2 to craft the preface and first SETTINGS
import h2.connection
conn = h2.connection.H2Connection()
conn.initiate_connection()
# The library automatically writes the preface + SETTINGS
raw = conn.data_to_send()
print(raw.hex())

The hex output starts with 504952202a20485454502f322e300d0a0d0a (the ASCII for the preface) followed by the 9-byte SETTINGS frame header.

Frame types (HEADERS, DATA, SETTINGS, PRIORITY, CONTINUATION, RST_STREAM, GOAWAY)

Every frame shares a common 9-byte header:

+-----------------------------------------------+
| Length (24) | Type (8) | Flags (8) | Stream Identifier (31) |
+-----------------------------------------------+

Below is a concise description of the most relevant frame types for security analysis.

HEADERS (0x01): Carries compressed header block. May be followed by CONTINUATION frames if the block is larger than the maximum frame size (default 16 KB).
DATA (0x00): Carries the request/response body. Subject to flow-control windows.
SETTINGS (0x04): Negotiates connection parameters. Each setting is a 16-bit identifier + 32-bit value.
PRIORITY (0x02): Assigns weight and dependency tree for stream scheduling. Abuse can lead to starvation attacks.
CONTINUATION (0x09): Extends a header block when HEADERS does not fit.
RST_STREAM (0x03): Abruptly terminates a single stream. Useful for DoS mitigation but also for smuggling if improperly handled.
GOAWAY (0x07): Gracefully shuts down the connection, indicating the last stream that was processed.

Sample HEADERS frame (hex) (simplified, after HPACK decoding):

00 00 15 01 04 00 00 00 01 88 85 86 84 41 0f 77 69 74 68 2d 63 6f 6f 6b 69 65 73

In practice, you rarely see raw frames; tools like nghttp or Wireshark decode them for you.

Stream multiplexing and flow control

Multiplexing eliminates the head-of-line blocking problem that plagued HTTP/1.1 pipelining. Each stream is independent; frames from different streams can be interleaved arbitrarily.

Flow control works on two levels:

Connection-level window: The total number of bytes the sender may transmit across all streams.
Stream-level window: The number of bytes the sender may transmit on a specific stream.

Both windows are advertised via WINDOW_UPDATE frames (type 0x08). If a window reaches zero, the sender must stop sending DATA frames until the receiver issues a WINDOW_UPDATE.

From a security perspective, malformed WINDOW_UPDATE values can be used to trigger integer-overflow bugs in naïve implementations, leading to DoS or memory corruption.

Practical demonstration with nghttp:

# Open a TLS connection and watch flow-control windows
nghttp -v https://example.com &
# In another terminal, send a large POST while limiting the window
nghttp -d largefile.bin -H "Content-Type: application/octet-stream" https://example.com/upload

Observe the WINDOW_UPDATE frames in the verbose output - they reveal how the server throttles the upload.

Header compression with HPACK

HPACK (RFC 7541) reduces header overhead by maintaining a static table of common header fields and a dynamic table that grows with each request/response. Headers are represented as indexed entries, literal name/value pairs, or incremental indexing.

Key security implications:

Compression side-channel attacks: Similar to BREACH on HTTP/1.1, HPACK can leak information if an attacker can observe the size of compressed header blocks.
Dynamic table poisoning: An attacker can inject crafted headers that cause the table to fill with malicious entries, potentially leading to mis-parsing on the other side.
Header splitting: Because headers are binary-encoded, an implementation that incorrectly splits on CRLF may be vulnerable to request smuggling when combined with legacy HTTP/1.1 parsers.

Example: Decoding an HPACK block with hpack Python library.

from hpack import Decoder
# Raw HPACK bytes captured from a HEADERS frame (escaped for JSON)
hpack_bytes = b"\x82\x86\x84\x41\x8c\x0f\x77\x69\x74\x68\x2d\x63\x6f\x6f\x6b\x69\x65\x73"
decoder = Decoder()
headers = decoder.decode(hpack_bytes)
print(headers)
# Expected output: [(b':method', b'GET'), (b':path', b'/'), (b'cookie', b'with-cookies')]

Understanding how to reconstruct the original header list is essential when you need to correlate a malicious request with its underlying HTTP/2 frames.

Differences between HTTP/1.1 and HTTP/2 relevant to smuggling

Request smuggling exploits ambiguities in how a front-end (e.g., a reverse proxy) parses the request line versus how the back-end does. HTTP/2 changes the playing field:

Binary framing vs. textual parsing: No more CRLF delimiters for the request line; the request method, scheme, path, and authority are separate pseudo-header fields (e.g., :method, :path).
Header ordering: HTTP/2 does not guarantee that pseudo-headers appear before regular headers, but many implementations enforce it. Mis-ordered pseudo-headers can be dropped by some parsers, causing inconsistencies.
Multiple HEADERS frames: A request can be split across several HEADERS/CONTINUATION frames. If a proxy only looks at the first HEADERS block, later pseudo-headers may be ignored, leading to request smuggling.
Stream IDs: Each request lives on its own stream. A malicious client can open many streams concurrently, confusing a proxy that expects a one-request-per-connection model.
Flow-control windows: An attacker can deliberately stall a stream, causing a downstream server to timeout while the upstream proxy still buffers data - another vector for smuggling.

In short, many classic smuggling tricks (duplicate Content-Length, Transfer-Encoding header) are irrelevant, but new vectors appear due to framing and HPACK handling.

Practical Examples

Example 1: Capturing HTTP/2 traffic with Wireshark

Start a TLS-terminated capture: tshark -i any -Y "http2" -w http2.pcap
Open the .pcap in Wireshark and apply the filter http2.
Expand a stream to see HEADERS, DATA, SETTINGS frames. Right-click a HEADERS frame → "Decode As" → "HPACK" to view the original header list.

This workflow is the basis for investigating suspicious streams that may be part of a smuggling attempt.

Example 2: Crafting a malformed SETTINGS frame to test a WAF

# Using nghttp2's h2load to send a custom SETTINGS payload
cat > bad_settings.bin <<EOF
ÿÿÿÿ
EOF
# Send the binary payload over TLS using openssl s_client
openssl s_client -connect target.com:443 -quiet < bad_settings.bin

The above sends a SETTINGS frame with an identifier of 0xffff (reserved) and a huge value, which some parsers mishandle, potentially causing them to ignore subsequent frames - a classic smuggling foothold.

Tools & Commands

nghttp / nghttpd - command-line HTTP/2 client/server for testing.
h2c - HTTP/2 clear-text (non-TLS) client useful for debugging.
Wireshark with the HTTP/2 dissector - visual inspection of frames.
h2spec - RFC-compliance testing suite; can be used to fuzz implementations.
mitmproxy --mode http2 - intercept and modify HTTP/2 traffic inline.

Sample command to list all streams in a live capture:

tshark -i eth0 -Y "http2" -T fields -e http2.streamid -c 20

Defense & Mitigation

Strict HPACK validation: Reject frames that exceed the advertised dynamic table size or contain illegal index values.
Enforce pseudo-header ordering: Ensure that :method, :scheme, :authority, and :path appear once and before regular headers.
Stream-level timeouts: Close streams that remain idle beyond a configurable threshold to avoid resource exhaustion.
Window-size limits: Cap the maximum connection-level window to a reasonable size (e.g., 1 MiB) to prevent memory-based DoS.
Upgrade-path sanitization: When supporting HTTP/1.1↔HTTP/2 upgrades (h2c), validate the HTTP2-Settings header carefully to avoid downgrade attacks.

Most modern web servers (nginx, Apache, Caddy) already implement these mitigations, but custom proxies or legacy appliances often lag behind.

Common Mistakes

Assuming a one-to-one mapping between TCP connections and HTTP requests - HTTP/2 multiplexes many requests on a single connection.
Parsing HEADERS frames as plain text without HPACK decoding, leading to missed pseudo-headers.
Ignoring CONTINUATION frames; some implementations treat missing CONTINUATION as a protocol error, while others silently concatenate.
Relying on Content-Length for request boundaries - HTTP/2 does not use this header for framing.
Disabling flow-control checks in development builds, which opens the door to flooding attacks.

Avoid these pitfalls by using a well-tested library (e.g., nghttp2) for any custom HTTP/2 handling.

Real-World Impact

Several high-profile incidents illustrate the importance of HTTP/2 security:

2020 Cloudflare bug: A malformed PRIORITY frame could cause a worker thread to panic, leading to temporary service disruption.
2021 GitHub request smuggling: An attacker leveraged a mis-ordered pseudo-header in an HTTP/2 request that a legacy load balancer interpreted as HTTP/1.1, bypassing input validation.
2023 CVE-2023-28640 (nghttp2): Integer overflow in WINDOW_UPDATE handling resulted in a denial-of-service when crafted frames were sent.

These cases show that even mature platforms can stumble on edge-case framing logic. Staying up-to-date with library patches and regularly running h2spec against your stack are practical defenses.

Practice Exercises

Capture and decode a real HTTP/2 session: Use nghttp -v pipe the output to <code>tshark, and identify the SETTINGS, HEADERS, and DATA frames.
Build a minimal HTTP/2 client using the hyper-h2 Python library that sends a GET request with a custom pseudo-header order. Observe how a mis-ordered header is treated by nghttpd.
Fuzz SETTINGS frames with h2spec -t and record which responses cause the server to close the connection. Document any differences between server implementations.
Simulate a smuggling scenario: Use mitmproxy to rewrite a HEADERS frame into two separate HEADERS frames (split pseudo-header and regular header). Verify whether the downstream component processes the request twice.

Document your findings in a short report - this exercise mirrors a typical penetration-testing workflow.

Summary

HTTP/2 replaces the textual line-oriented model of HTTP/1.1 with a binary, frame-based protocol that introduces multiplexing, flow control, and HPACK compression. Security professionals must grasp the connection preface, SETTINGS exchange, and the full set of frame types because many classic attacks (e.g., request smuggling) manifest differently under these rules. Proper inspection tools, strict parsing, and up-to-date libraries are the pillars of a resilient HTTP/2 deployment.

Key takeaways:

Every request lives on a stream; frames can be interleaved arbitrarily.
HPACK compression is both a performance win and a potential side-channel.
Mis-ordered pseudo-headers, malformed SETTINGS, and abused flow-control windows are the most common vectors for HTTP/2-specific smuggling.
Use nghttp, Wireshark, and h2spec to validate implementations.

HTTP/2 Protocol Overview & Frame Structure - Introductory Guide