Use of set_tunnel with default port results in incorrect post value in host header

[demianbrecht]

BPO	22095
Nosy	@orsenthil, @berkerpeksag, @serhiy-storchaka, @demianbrecht
Files	issue22095.patch issue22095_1.patch issue22095_2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2014-12-12.07:47:00.863>
created_at = <Date 2014-07-28.18:09:43.881>
labels = ['type-bug', 'library']
title = 'Use of set_tunnel with default port results in incorrect post value in host header'
updated_at = <Date 2015-05-28.19:44:32.545>
user = 'https://github.com/demianbrecht'

bugs.python.org fields:

activity = <Date 2015-05-28.19:44:32.545>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2014-12-12.07:47:00.863>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2014-07-28.18:09:43.881>
creator = 'demian.brecht'
dependencies = []
files = ['36140', '37309', '37419']
hgrepos = []
issue_num = 22095
keywords = ['patch']
message_count = 10.0
messages = ['224175', '224991', '231308', '231834', '232502', '232508', '232513', '232514', '244341', '244342']
nosy_count = 6.0
nosy_names = ['orsenthil', 'nikratio', 'python-dev', 'berker.peksag', 'serhiy.storchaka', 'demian.brecht']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue22095'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5']

[demianbrecht]

Creating this bug for clarity, but was encountered while investigating 22041 (now set as not a bug).

When using set_tunnel with default port, the port value in the host header is set to "None":

"send: b'POST [PATH] HTTP/1.1\r\nHost: [HOST]:None\r\nAccept-Encoding: identity\r\nContent-Length: 41\r\nAccept: text/plain\r\nContent-type: application/x-www-form-urlencoded\r\n\r\n[FORM_DATA]'"

This issue seems to have been introduced in 3.4 when _tunnel_host and _tunnel_port were introduced to decouple the destination host and port from the proxy host and port.

[demianbrecht]

Bump for review

[serhiy-storchaka]

If call _get_hostport() in set_tunnel() then it should be removed in _tunnel().

As for tests, it would be better do not rely on implementation details. Instead you can monkey-patch the send() method of of HTTPConnection instance and check passed argument.

[demianbrecht]

Thanks Serhiy, new patch addresses your comments.

[serhiy-storchaka]

See also comments on Rietveld (e-mail notification likely fell in spam).

[demianbrecht]

Thanks for the ping Serhiy, indeed the review notification email was sitting in spam. New patch addressing review comments as well as rectifying my own silliness.

[python-dev]

New changeset 676d6bcfc031 by Serhiy Storchaka in branch '3.4':
Issue bpo-22095: Fixed HTTPConnection.set_tunnel with default port. The port
https://hg.python.org/cpython/rev/676d6bcfc031

New changeset ebe2072e5472 by Serhiy Storchaka in branch 'default':
Issue bpo-22095: Fixed HTTPConnection.set_tunnel with default port. The port
https://hg.python.org/cpython/rev/ebe2072e5472

[serhiy-storchaka]

Thank you for your contribution Demian.

[python-dev]

New changeset 3b6e0720a69d by Serhiy Storchaka in branch '2.7':
Issue bpo-22095: Fixed HTTPConnection.set_tunnel with default port. The port
https://hg.python.org/cpython/rev/3b6e0720a69d

[serhiy-storchaka]

Backported to 2.7 because this is needed to support proxy with a port in urllib/urllib2 as documented. See bpo-24311.