[security][CVE-2020-8492] Denial of service in urllib.request.AbstractBasicAuthHandler

[vstinner]

BPO	39503
Nosy	@vstinner, @larryhastings, @ned-deily, @mgorny, @koobs, @miss-islington, @ware, @tapakund, @bcaller
PRs	bpo-39503: CVE-2020-8492: Fix AbstractBasicAuthHandler #18284 [3.8] bpo-39503: CVE-2020-8492: Fix AbstractBasicAuthHandler (GH-18284) #19291 [3.7] bpo-39503: CVE-2020-8492: Fix AbstractBasicAuthHandler (GH-18284) #19292 [3.8] bpo-39503: CVE-2020-8492: Fix AbstractBasicAuthHandler (GH-18284) #19296 [3.7] bpo-39503: CVE-2020-8492: Fix AbstractBasicAuthHandler (GH-18284) #19297 [3.6] bpo-39503: CVE-2020-8492: Fix AbstractBasicAuthHandler #19299 [3.5] bpo-39503: CVE-2020-8492: Fix AbstractBasicAuthHandler #19301 [2.7] bpo-39503: CVE-2020-8492: Fix AbstractBasicAuthHandler #19302 [3.6] bpo-39503: CVE-2020-8492: Fix AbstractBasicAuthHandler (GH-18284) #19304 [3.5] bpo-39503: CVE-2020-8492: Fix AbstractBasicAuthHandler (GH-18284) #19305
Files	bench_parser.py bench_parser2.py: Benchmark

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-06-20.08:33:12.125>
created_at = <Date 2020-01-30.15:11:29.386>
labels = ['type-security', '3.8', '3.9', '3.10', '3.11', '3.7', 'library']
title = '[security][CVE-2020-8492] Denial of service in urllib.request.AbstractBasicAuthHandler'
updated_at = <Date 2021-09-14.10:39:30.313>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2021-09-14.10:39:30.313>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-06-20.08:33:12.125>
closer = 'larry'
components = ['Library (Lib)']
creation = <Date 2020-01-30.15:11:29.386>
creator = 'vstinner'
dependencies = []
files = ['49016', '49023']
hgrepos = []
issue_num = 39503
keywords = ['patch']
message_count = 17.0
messages = ['361072', '361073', '361081', '361343', '364996', '365335', '365538', '365541', '365542', '365578', '365579', '365663', '371921', '401762', '401766', '401768', '401770']
nosy_count = 11.0
nosy_names = ['vstinner', 'larry', 'ned.deily', 'mgorny', 'koobs', 'miss-islington', 'ware', 'tapakund', 'Anselmo Melo', 'bc', 'sxt1001']
pr_nums = ['18284', '19291', '19292', '19296', '19297', '19299', '19301', '19302', '19304', '19305']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'security'
url = 'https://bugs.python.org/issue39503'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10', 'Python 3.11']

[vstinner]

Copy of an email received on the Python Security Response team, 9 days ago. I consider that it's not worth it to have an embargo on this vulnerability, so I make it public.

Hi there,

I believe I've found a denial-of-service (DoS) bug in
urllib.request.AbstractBasicAuthHandler. To start, I'm operating on some
background information from this document: HTTP authentication
<https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication\>. The bug
itself is a ReDoS <https://www.regular-expressions.info/redos.html\> bug
causing catastrophic backtracking. To reproduce the issue we can use the
following code:

from urllib.request import AbstractBasicAuthHandler
auth_handler = AbstractBasicAuthHandler()
auth_handler.http_error_auth_reqed(
    'www-authenticate',
    'unused',
    'unused',
    {
        'www-authenticate': 'Basic ' + ',' * 64 + ' ' + 'foo' + ' ' +
'realm'
    }
)

The issue itself is in the following regular expression:

rx = re.compile('(?:.*,)*[ \t]*([^ \t]+)[ \t]+'
                'realm=(["\']?)([^"\']*)\\2', re.I)

In particular, the (?:.*,)* portion. Since "." and "," overlap and there
are nested quantifiers we can cause catastrophic backtracking by repeating
a comma. Note that since AbstractBasicAuthHandler is vulnerable, then both
HTTPBasicAuthHandler and ProxyBasicAuthHandler are as well because they
call http_error_auth_reqed. Building from the HTTP authentication document
above, this means a server can send a specially crafted header along with
an HTTP 401 or HTTP 407 and cause a DoS on the client.

I won't speculate on the severity of the issue too much - you will surely
understand the impact better than I will. Although, the fact that this is
client-side as opposed to server-side appears to reduce the severity,
however the fact that it's a security-sensitive context (HTTP
authentication) may raise the severity.

One possible fix would be changing the rx expression to the following:

rx = re.compile('(?:[^,]*,)*[ \t]*([^ \t]+)[ \t]+'
                'realm=(["\']?)([^"\']*)\\2', re.I)

This removes the character overlap in the nested quantifier and thus
negates the catastrophic backtracking.

Let me know if you have any questions or what the next steps are from here.
Thanks for supporting Python security!

--
Matt Schwager

[vstinner]

I added this vulnerability to the following page to track fixes in all Python supported branches:
https://python-security.readthedocs.io/vuln/urllib-basic-auth-regex.html

[vstinner]

CVE-2020-8492 has been assigned to this vulnerability:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=2020-8492