xmlrpc: gzip_decode has unlimited read()

[tiran]

BPO	16043
Nosy	@loewis, @warsaw, @birkenfeld, @doko42, @larryhastings, @tiran, @matejcik, @benjaminp, @florentx, @vadmium, @JimJJewett, @serhiy-storchaka
Files	xmlrpc_gzip_27.patch xmlrpc_gzip_27_parameter.patch xmlrpc_gzip_27_parameter.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/tiran'
closed_at = <Date 2014-12-06.01:17:14.489>
created_at = <Date 2012-09-25.10:52:06.421>
labels = ['release-blocker', 'library', 'performance']
title = 'xmlrpc: gzip_decode has unlimited read()'
updated_at = <Date 2014-12-06.01:36:14.705>
user = 'https://github.com/tiran'

bugs.python.org fields:

activity = <Date 2014-12-06.01:36:14.705>
actor = 'python-dev'
assignee = 'christian.heimes'
closed = True
closed_date = <Date 2014-12-06.01:17:14.489>
closer = 'python-dev'
components = ['Library (Lib)']
creation = <Date 2012-09-25.10:52:06.421>
creator = 'christian.heimes'
dependencies = []
files = ['28796', '37343', '37344']
hgrepos = []
issue_num = 16043
keywords = ['patch']
message_count = 16.0
messages = ['171246', '171247', '180296', '182087', '182093', '182138', '182203', '185061', '196857', '200345', '207176', '213976', '231989', '231992', '232231', '232235']
nosy_count = 15.0
nosy_names = ['loewis', 'barry', 'georg.brandl', 'doko', 'larry', 'christian.heimes', 'schmir', 'matejcik', 'benjamin.peterson', 'Arfrever', 'flox', 'python-dev', 'martin.panter', 'Jim.Jewett', 'serhiy.storchaka']
pr_nums = []
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue16043'
versions = ['Python 3.1', 'Python 2.7', 'Python 3.2', 'Python 3.3', 'Python 3.4']

[tiran]

The xmlrpc client library is the only stdlib module that has a gzip decompression handler for compressed HTTP streams. The gzip_decode() function decompresses HTTP bodies that are compressed and sent with Accept-Encoding: x-gzip.

A malicious server can send a specially prepared HTTP request that can consume lots of memory. For example 1 GB of \0 bytes is less than 1 MB of gzip data.

Suggestion:
The gzip_decode() should only decode a sane amount of bytes (for example 50 MB) and raise an exception when more data is to be read.

[tiran]

Also see bpo-15955

According to Nadeem it's not (easily) possible to detect how large the output is going to be.

[tiran]

The attached patch adds a limitation to xmlrpclib.gzip_decode().

[tiran]

IMHO the patch should also limit the maximum amount of read bytes in Transport.parse_response(). Do you agree?

[serhiy-storchaka]

I think instead of global variable it will be better to add an optional parameter for gzip_decode() (with a sane default value) and related functions. Or at least in additional to it.

[tiran]

+1 for a keyword argument

I also have to add a limit to GzipDecodedResponse().

Python 2.6 and 3.1 are not affected by the issue. The problematic code was added in 2.7 and 3.2.

[tiran]

CVE-2013-1753 gzip bomb and unbound read DoS vulnerabilities in Python's xmlrpc library

[benjaminp]

Not blocking 2.7.4 as discussed on mailing list.

[warsaw]

blocker for 2.6.9