socket lib beahavior change in 3.6.4

[skn78]

BPO	32394
Nosy	@pfmoore, @tjguk, @ned-deily, @njsmith, @asvetlov, @methane, @zware, @zooba, @srinivasreddy, @animalize, @miss-islington
PRs	bpo-32394: Remove some TCP options on old version Windows. #5468 bpo-32394: Remove some TCP options on old version Windows. #5523 [3.6] bpo-32394: Remove some TCP options on older version Windows. (GH-5523) #5585 [3.7] bpo-32394: Remove some TCP options on old version Windows. (GH-5523) #5910
Files	winsdk_watchdog.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/zooba'
closed_at = <Date 2018-02-26.19:00:09.160>
created_at = <Date 2017-12-21.00:12:44.238>
labels = ['type-bug', '3.8', 'release-blocker', '3.7', 'library', 'OS-windows']
title = 'socket lib beahavior change in 3.6.4'
updated_at = <Date 2018-02-26.19:00:09.159>
user = 'https://bugs.python.org/skn78'

bugs.python.org fields:

activity = <Date 2018-02-26.19:00:09.159>
actor = 'steve.dower'
assignee = 'steve.dower'
closed = True
closed_date = <Date 2018-02-26.19:00:09.160>
closer = 'steve.dower'
components = ['Library (Lib)', 'Windows']
creation = <Date 2017-12-21.00:12:44.238>
creator = 'skn78'
dependencies = []
files = ['47431']
hgrepos = []
issue_num = 32394
keywords = ['patch', '3.6regression']
message_count = 49.0
messages = ['308837', '308864', '308883', '308885', '308888', '308900', '308901', '308902', '308904', '308906', '308943', '311172', '311197', '311400', '311401', '311414', '311465', '311466', '311487', '311502', '311505', '311506', '311580', '311619', '311620', '311621', '311641', '311642', '311644', '311652', '311653', '311654', '311658', '311713', '311714', '311718', '311725', '311753', '311754', '311774', '311811', '311858', '311972', '312836', '312915', '312941', '312942', '312943', '312944']
nosy_count = 13.0
nosy_names = ['paul.moore', 'tim.golden', 'ned.deily', 'njs', 'asvetlov', 'methane', 'zach.ware', 'steve.dower', 'thatiparthy', 'malin', 'tjguk', 'skn78', 'miss-islington']
pr_nums = ['5468', '5523', '5585', '5910']
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue32394'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

[skn78]

On Windows, python 3.6.3 code "hasattr(socket, 'TCP_KEEPCNT')" gives False, python 3.6.4 gives True. This behavior breaks many libraries that i use.

[srinivasreddy]

dtdev@dtdev-centos $ python3
Python 3.6.3 (default, Oct 11 2017, 18:17:01) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-18)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> hasattr(socket, 'TCP_KEEPCNT')
True
>>> 

I have linux system. Above is the output i got.

[zooba]

Which libraries break? And which version of Windows are you running on?

If TCP_KEEPCNT was only added recently (perhaps in an SDK update) then it may not really be available on all versions.

(+Ned for awareness of the regression - not sure we need to respin immediately, but let's figure it out asap)

[tjguk]

It's a compile-time option in socketmodule.c.

https://github.com/python/cpython/blob/3.6/Modules/socketmodule.c#L7466

The MSDN page suggests that it was added for Win10:

https://msdn.microsoft.com/en-us/library/windows/desktop/ms738596(v=vs.85).aspx

Is it possible that the build machine has changed its OS/SDK between
building 3.6.3 and 3.6.4?

[zooba]

Yeah, I updated the build machine before doing 3.6.4rc1 (I never update between rc and final releases). I'm intrigued to see why it breaks libraries though - typically unsupported enum values like this are silently ignored on older Windows versions.

[skn78]

My OS version is Windows 7 x64.
I ran the script on the same computer, but with different versions of the python:

import socket
import platform

print('1) OS Info: ', platform.architecture(), platform.platform())
print('2) Python Info: ', platform.python_build(), platform.python_compiler())
print('3) TCP_KEEPCNT = ', hasattr(socket, 'TCP_KEEPCNT') )

Result for python 3.6.3:

1. OS Info: ('64bit', 'WindowsPE') Windows-7-6.1.7601-SP1
2. Python Info: ('v3.6.3:2c5fed8', 'Oct 3 2017 18:11:49') MSC v.1900 64 bit (AMD64)
3. TCP_KEEPCNT = False

Result for python 3.6.4:

1. OS Info: ('64bit', 'WindowsPE') Windows-7-6.1.7601-SP1
2. Python Info: ('v3.6.4:d48eceb', 'Dec 19 2017 06:54:40') MSC v.1900 64 bit (AMD64)
3. TCP_KEEPCNT = True

[zooba]

Yes, I recognise that the change happened. I don't understand what breaks as a result.

You said it breaks "many libraries" - can you name some of them and provide sample code?

[skn78]

websocket-client 0.44.0
https://pypi.python.org/pypi/websocket-client/0.44.0

My script gives the following Erroe:

File "C:\Program Files\Python36\lib\site-packages\websocket\_http.py", line 108, in _open_socket
sock.setsockopt(*opts)
OSError: [WinError 10042] Для вызова getsockopt или setsockopt был указан неизвестный, недопустимый или неподдерживаемый параметр
или уровень

[zooba]

Okay, so it looks like we don't have any prior art to conditionally omit constants from _socket based on OS level, and nothing in the setsockopt() doc suggests that it may fail like this.

So we either need to explicitly exclude this symbol on Windows (at least until we drop support for pre-Windows 10 versions) or silently ignore setsockopt errors for future good arguments. I'm inclined to do the former - other opinions?

[tjguk]

Excluding for now seems like a simple option. (Basically a reversion to
previous behaviour). And allows us easily to include again later easily.

Messing with setsockopt seems a little more risky.

In short: I'm with you -- exclude for now.

[skn78]

I suggest inserting the following code into socket.py:

if hasattr(sys, 'getwindowsversion') and sys.getwindowsversion()[0] < 10:
    del socket.TCP_KEEPCNT
    del socket.TCP_FASTOPEN