select.poll.poll fails on BSDs with arbitrary negative timeouts

[volans]

BPO	31334
Nosy	@ronaldoussoren, @vstinner, @ned-deily, @serhiy-storchaka, @koobs, @volans-
PRs	bpo-31334: Fix timeout in select.poll.poll() #3277 [3.6] bpo-31334: Fix timeout in select.poll.poll() (GH-3277) #4033 [2.7] bpo-31334: Fix timeout in select.poll.poll() (GH-3277) #4034

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-10-18.12:05:03.263>
created_at = <Date 2017-09-03.22:21:42.443>
labels = ['extension-modules', 'OS-mac', 'type-bug', '3.7']
title = 'select.poll.poll fails on BSDs with arbitrary negative timeouts'
updated_at = <Date 2017-10-18.12:05:18.503>
user = 'https://github.com/volans'

bugs.python.org fields:

activity = <Date 2017-10-18.12:05:18.503>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2017-10-18.12:05:03.263>
closer = 'serhiy.storchaka'
components = ['Extension Modules', 'macOS', 'FreeBSD']
creation = <Date 2017-09-03.22:21:42.443>
creator = 'Riccardo Coccioli'
dependencies = []
files = []
hgrepos = []
issue_num = 31334
keywords = ['patch']
message_count = 7.0
messages = ['301202', '301308', '304472', '304473', '304517', '304575', '304576']
nosy_count = 6.0
nosy_names = ['ronaldoussoren', 'vstinner', 'ned.deily', 'serhiy.storchaka', 'koobs', 'Riccardo Coccioli']
pr_nums = ['3277', '4033', '4034']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue31334'
versions = ['Python 2.7', 'Python 3.6', 'Python 3.7']

[volans]

According to the Python documentation for the 'poll.poll([timeout])' method in the 'select' module, any negative value for the 'timeout' parameter is valid and should have the same behaviour [1]:
"If timeout is omitted, negative, or None, the call will block until there is an event for this poll object."

Unfortunately, unlike the Linux, on many other OSes, including, but not limited to, macOS and {Free,Open,Net}BSD, the 'poll()' system call requires that the 'timeout' parameter is a non-negative integer or exactly -1 (sometimes defined as INFTIM). Any other negative value throws an error, see [2], [3], [4] and [5].

This is a snippet of code to reproduce the error:
#-----

import select

p = select.poll()
p.poll(-100)
#

---

Expected behaviour: block until there is an event for the poll object, in this case block indefinitely
Current behaviour on macOS and FreeBSD: OSError: [Errno 22] Invalid argument

I was able to reproduce the error on:

• macOS Sierra 10.12.6 with those Python versions: 3.3.6, 3.4.6, 3.5.3, 3.6.2, 3.7.0a0 (heads/master:2ef37607b7)
• FreeBSD 11.1 with Python 3.7.0a0 (heads/master:2ef37607b7)

On Linux this doesn't happen because the 'poll()' system call accept any negative value to wait indefinitely, see [6].
To adhere with the Python documentation described behaviour, I'm sending a pull request to propose to force the 'timeout' value passed to the 'poll()' system call to be exactly -1 (or INFTIM where defined) when a negative value is given.
This will not change the current behaviour on Linux and will have the behaviour described in the documentation on other OSes where is currently failing with an error.

[1] https://docs.python.org/3/library/select.html#poll-objects
[2] https://www.freebsd.org/cgi/man.cgi?poll
[3] https://man.openbsd.org/poll.2
[4] http://netbsd.gw.com/cgi-bin/man-cgi/man?poll
[5] From macOS 'man poll': "If timeout is greater than zero, it specifies a maximum interval (in milliseconds) to wait for any file descriptor to become ready. If timeout is zero, then poll() will return without blocking. If the value of timeout is -1, the poll blocks indefinitely."
[6] http://man7.org/linux/man-pages/man2/poll.2.html

[volans]

This can actually be reproduced with Python 2.7 too (thanks @thiell to let me know). At first I thought that it was not because it doesn't repro with the stock macOS-shipped Python 2.7.10 on macOS Sierra 10.12.6, where the select.poll() is not available at all, see below.

Updated list of version where I was able to reproduce the error:

• macOS Sierra 10.12.6 with those Python versions: 2.7.10, 2.7.13, 3.3.6, 3.4.6, 3.5.3, 3.6.2, 3.7.0a0 (heads/master:2ef37607b7)
• FreeBSD 11.1 with those Python versions: 2.7.13, 3.7.0a0 (heads/master:2ef37607b7)

For reference, the repro code executed with the stock macOS-shipped Python:
#------

$ /usr/bin/python
Python 2.7.10 (default, Feb  7 2017, 00:08:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import select
>>> p = select.poll()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'poll'
#

---

If the PR 3277 that I've sent against the master branch with the fix will be accepted, I'm ready to send additional PRs to backport the fix in all affected versions.

[serhiy-storchaka]

Shouldn't epoll_poll() be fixed too? Only -1 is documented as an infinity timeout in epoll_wait (2).

[volans]

Although it's documented as -1 in Linux man page [1], from my quick tests I was not able to get any error with negative values different from -1 and it seems to wait indefinitely as expected. Looking also at its implementation in [2], it doesn't seem to differentiate between negative values. It could be argued that is implementation dependent at the moment and the behaviour might change in the future.
But, on a related note, the Python documentation doesn't say much either as what are acceptable values for the timeout parameter, see [3].

So at the moment there isn't any discrepancy between the Python documentation and the current behaviour IMHO, but I'm happy to open a separate task and send a PR if you think this should be improved/fixed too.

[1] http://man7.org/linux/man-pages/man2/epoll_wait.2.html
[2] http://elixir.free-electrons.com/linux/latest/source/fs/eventpoll.c#L1754
[3] https://docs.python.org/3.7/library/select.html

[vstinner]

New changeset 6cfa927 by Victor Stinner (Riccardo Coccioli) in branch 'master':
bpo-31334: Fix timeout in select.poll.poll() (GH-3277)
6cfa927

[serhiy-storchaka]

New changeset 27b951c by Serhiy Storchaka (Riccardo Coccioli) in branch '2.7':
[2.7] bpo-31334: Fix timeout in select.poll.poll() (GH-3277) (bpo-4034)
27b951c

[serhiy-storchaka]

New changeset 97abcab by Serhiy Storchaka (Miss Islington (bot)) in branch '3.6':
[3.6] bpo-31334: Fix timeout in select.poll.poll() (GH-3277) (bpo-4033)
97abcab