Skip to content

gh-131724: Add a new max_response_headers param to HTTP/HTTPSConnection #136814

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jul 20, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 19 additions & 3 deletions Doc/library/http.client.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ The module provides the following classes:


.. class:: HTTPConnection(host, port=None[, timeout], source_address=None, \
blocksize=8192)
blocksize=8192, max_response_headers=None)

An :class:`HTTPConnection` instance represents one transaction with an HTTP
server. It should be instantiated by passing it a host and optional port
Expand All @@ -46,7 +46,9 @@ The module provides the following classes:
The optional *source_address* parameter may be a tuple of a (host, port)
to use as the source address the HTTP connection is made from.
The optional *blocksize* parameter sets the buffer size in bytes for
sending a file-like message body.
sending a file-like message body. The optional *max_response_headers*
parameter sets the maximum number of allowed response headers to help
prevent denial-of-service attacks, otherwise the default value (100) is used.

For example, the following calls all create instances that connect to the server
at the same host and port::
Expand All @@ -66,10 +68,13 @@ The module provides the following classes:
.. versionchanged:: 3.7
*blocksize* parameter was added.

.. versionchanged:: next
*max_response_headers* parameter was added.


.. class:: HTTPSConnection(host, port=None, *[, timeout], \
source_address=None, context=None, \
blocksize=8192)
blocksize=8192, max_response_headers=None)

A subclass of :class:`HTTPConnection` that uses SSL for communication with
secure servers. Default port is ``443``. If *context* is specified, it
Expand Down Expand Up @@ -109,6 +114,9 @@ The module provides the following classes:
The deprecated *key_file*, *cert_file* and *check_hostname* parameters
have been removed.

.. versionchanged:: next
*max_response_headers* parameter was added.


.. class:: HTTPResponse(sock, debuglevel=0, method=None, url=None)

Expand Down Expand Up @@ -416,6 +424,14 @@ HTTPConnection Objects
.. versionadded:: 3.7


.. attribute:: HTTPConnection.max_response_headers

The maximum number of allowed response headers to help prevent denial-of-service
attacks. By default, the maximum number of allowed headers is set to 100.

.. versionadded:: next


As an alternative to using the :meth:`~HTTPConnection.request` method described above, you can
also send your request step by step, by using the four functions below.

Expand Down
10 changes: 10 additions & 0 deletions Doc/whatsnew/3.15.rst
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,16 @@ difflib
(Contributed by Jiahao Li in :gh:`134580`.)


http.client
-----------

* A new *max_response_headers* keyword-only parameter has been added to
:class:`~http.client.HTTPConnection` and :class:`~http.client.HTTPSConnection`
constructors. This parameter overrides the default maximum number of allowed
response headers.
(Contributed by Alexander Enrique Urieles Nieto in :gh:`131724`.)


math
----

Expand Down
40 changes: 25 additions & 15 deletions Lib/http/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -209,22 +209,24 @@ def getallmatchingheaders(self, name):
lst.append(line)
return lst

def _read_headers(fp):
def _read_headers(fp, max_headers):
"""Reads potential header lines into a list from a file pointer.

Length of line is limited by _MAXLINE, and number of
headers is limited by _MAXHEADERS.
headers is limited by max_headers.
"""
headers = []
if max_headers is None:
max_headers = _MAXHEADERS
while True:
line = fp.readline(_MAXLINE + 1)
if len(line) > _MAXLINE:
raise LineTooLong("header line")
headers.append(line)
if len(headers) > _MAXHEADERS:
raise HTTPException("got more than %d headers" % _MAXHEADERS)
if line in (b'\r\n', b'\n', b''):
break
headers.append(line)
if len(headers) > max_headers:
raise HTTPException(f"got more than {max_headers} headers")
return headers

def _parse_header_lines(header_lines, _class=HTTPMessage):
Expand All @@ -241,10 +243,10 @@ def _parse_header_lines(header_lines, _class=HTTPMessage):
hstring = b''.join(header_lines).decode('iso-8859-1')
return email.parser.Parser(_class=_class).parsestr(hstring)

def parse_headers(fp, _class=HTTPMessage):
def parse_headers(fp, _class=HTTPMessage, *, _max_headers=None):
"""Parses only RFC2822 headers from a file pointer."""

headers = _read_headers(fp)
headers = _read_headers(fp, _max_headers)
return _parse_header_lines(headers, _class)


Expand Down Expand Up @@ -320,7 +322,7 @@ def _read_status(self):
raise BadStatusLine(line)
return version, status, reason

def begin(self):
def begin(self, *, _max_headers=None):
if self.headers is not None:
# we've already started reading the response
return
Expand All @@ -331,7 +333,7 @@ def begin(self):
if status != CONTINUE:
break
# skip the header from the 100 response
skipped_headers = _read_headers(self.fp)
skipped_headers = _read_headers(self.fp, _max_headers)
if self.debuglevel > 0:
print("headers:", skipped_headers)
del skipped_headers
Expand All @@ -346,7 +348,9 @@ def begin(self):
else:
raise UnknownProtocol(version)

self.headers = self.msg = parse_headers(self.fp)
self.headers = self.msg = parse_headers(
self.fp, _max_headers=_max_headers
)

if self.debuglevel > 0:
for hdr, val in self.headers.items():
Expand Down Expand Up @@ -864,7 +868,7 @@ def _get_content_length(body, method):
return None

def __init__(self, host, port=None, timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
source_address=None, blocksize=8192):
source_address=None, blocksize=8192, *, max_response_headers=None):
self.timeout = timeout
self.source_address = source_address
self.blocksize = blocksize
Expand All @@ -877,6 +881,7 @@ def __init__(self, host, port=None, timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
self._tunnel_port = None
self._tunnel_headers = {}
self._raw_proxy_headers = None
self.max_response_headers = max_response_headers

(self.host, self.port) = self._get_hostport(host, port)

Expand Down Expand Up @@ -969,7 +974,7 @@ def _tunnel(self):
try:
(version, code, message) = response._read_status()

self._raw_proxy_headers = _read_headers(response.fp)
self._raw_proxy_headers = _read_headers(response.fp, self.max_response_headers)

if self.debuglevel > 0:
for header in self._raw_proxy_headers:
Expand Down Expand Up @@ -1426,7 +1431,10 @@ def getresponse(self):

try:
try:
response.begin()
if self.max_response_headers is None:
response.begin()
else:
response.begin(_max_headers=self.max_response_headers)
except ConnectionError:
self.close()
raise
Expand Down Expand Up @@ -1457,10 +1465,12 @@ class HTTPSConnection(HTTPConnection):

def __init__(self, host, port=None,
*, timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
source_address=None, context=None, blocksize=8192):
source_address=None, context=None, blocksize=8192,
max_response_headers=None):
super(HTTPSConnection, self).__init__(host, port, timeout,
source_address,
blocksize=blocksize)
blocksize=blocksize,
max_response_headers=max_response_headers)
if context is None:
context = _create_https_context(self._http_vsn)
self._context = context
Expand Down
46 changes: 46 additions & 0 deletions Lib/test/test_httplib.py
Original file line number Diff line number Diff line change
Expand Up @@ -386,6 +386,52 @@ def test_headers_debuglevel(self):
self.assertEqual(lines[2], "header: Second: val1")
self.assertEqual(lines[3], "header: Second: val2")

def test_max_response_headers(self):
max_headers = client._MAXHEADERS + 20
headers = [f"Name{i}: Value{i}".encode() for i in range(max_headers)]
body = b"HTTP/1.1 200 OK\r\n" + b"\r\n".join(headers)

with self.subTest(max_headers=None):
sock = FakeSocket(body)
resp = client.HTTPResponse(sock)
with self.assertRaisesRegex(
client.HTTPException, f"got more than 100 headers"
):
resp.begin()

with self.subTest(max_headers=max_headers):
sock = FakeSocket(body)
resp = client.HTTPResponse(sock)
resp.begin(_max_headers=max_headers)

def test_max_connection_headers(self):
max_headers = client._MAXHEADERS + 20
headers = (
f"Name{i}: Value{i}".encode() for i in range(max_headers - 1)
)
body = (
b"HTTP/1.1 200 OK\r\n"
+ b"\r\n".join(headers)
+ b"\r\nContent-Length: 12\r\n\r\nDummy body\r\n"
)

with self.subTest(max_headers=None):
conn = client.HTTPConnection("example.com")
conn.sock = FakeSocket(body)
conn.request("GET", "/")
with self.assertRaisesRegex(
client.HTTPException, f"got more than {client._MAXHEADERS} headers"
):
response = conn.getresponse()

with self.subTest(max_headers=None):
conn = client.HTTPConnection(
"example.com", max_response_headers=max_headers
)
conn.sock = FakeSocket(body)
conn.request("GET", "/")
response = conn.getresponse()
response.read()

class HttpMethodTests(TestCase):
def test_invalid_method_names(self):
Expand Down
1 change: 1 addition & 0 deletions Misc/ACKS
Original file line number Diff line number Diff line change
Expand Up @@ -1954,6 +1954,7 @@ Adnan Umer
Utkarsh Upadhyay
Roger Upole
Daniel Urban
Alexander Enrique Urieles Nieto
Matthias Urlichs
Michael Urman
Hector Urtubia
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
In :mod:`http.client`, a new *max_response_headers* keyword-only parameter has been
added to :class:`~http.client.HTTPConnection` and :class:`~http.client.HTTPSConnection`
constructors. This parameter sets the maximum number of allowed response headers,
helping to prevent denial-of-service attacks.
Loading
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy