Issues with kqueue

[serhiy-storchaka]

BPO	31893
Nosy	@vstinner, @serhiy-storchaka, @koobs
PRs	bpo-31893: Fixed select.kqueue(). #4166 [3.6] bpo-31893: Fixed select.kqueue(). (GH-4166) #4190 [2.7] bpo-31893: Fixed select.kqueue(). (GH-4166) #4193 bpo-31893: Fix errors in b9052a0f91d2e83bbc27267247a5920c82b242a3. #4196 [3.6] bpo-31893: Fix errors in b9052a0f91d2e83bbc27267247a5920c82b242a3. (GH-4196) #4201 [2.7] bpo-31893: Fix errors in b9052a0f91d2e83bbc27267247a5920c82b242a3. (GH-4196) #4202 [2.7] bpo-31893: Fix a backporting error in 8cbf4e10646c3f5b8f0d274c2d7dea5bb6305f57. #4219 [2.7] bpo-31893: Fix a test for select.kevent. #4349

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2017-11-09.17:18:30.162>
created_at = <Date 2017-10-29.11:44:41.930>
labels = ['extension-modules', 'type-bug', '3.7']
title = 'Issues with kqueue'
updated_at = <Date 2017-11-09.17:18:30.161>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2017-11-09.17:18:30.161>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2017-11-09.17:18:30.162>
closer = 'serhiy.storchaka'
components = ['Extension Modules', 'FreeBSD']
creation = <Date 2017-10-29.11:44:41.930>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 31893
keywords = ['patch']
message_count = 20.0
messages = ['305187', '305290', '305294', '305295', '305300', '305305', '305309', '305312', '305325', '305326', '305329', '305385', '305948', '305949', '305950', '305958', '305964', '305965', '305966', '305971']
nosy_count = 3.0
nosy_names = ['vstinner', 'serhiy.storchaka', 'koobs']
pr_nums = ['4166', '4190', '4193', '4196', '4201', '4202', '4219', '4349']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue31893'
versions = ['Python 2.7', 'Python 3.6', 'Python 3.7']

[serhiy-storchaka]

In Modules/selectmodule.c it is assumed that the kevent structure is defined on FreeBSD and NetBSD as:

     struct kevent {
             uintptr_t ident;
             short     filter;
             u_short   flags;
             u_int     fflags;
             intptr_t  data;
             uintptr_t udata;
     };

and on OpenBSD as:

     struct kevent {
             u_int     ident;
             short     filter;
             u_short   flags;
             u_int     fflags;
             intptr_t  data;
             int       udata;
     };

Actually it is defined on FreeBSD as:

     struct kevent {
             uintptr_t ident;
             short     filter;
             u_short   flags;
             u_int     fflags;
             intptr_t  data;
             void      *udata;
     };

On OpenBSD as:

     struct kevent {
             uintptr_t ident;
             short     filter;
             u_short   flags;
             u_int     fflags;
             int64_t   data;
             void      *udata;
     };

And on NetBSD as:

     struct kevent {
             uintptr_t ident;
             uint32_t  filter;
             uint32_t  flags;
             uint32_t  fflags;
             int64_t   data;
             intptr_t  udata;
     };

Other issues are related to rich comparison. Due to integer overflows the ordering is not transitive. The rich comparison protocol is not properly supported, comparing a kevent_event object with a non-kevent_event object don't falls back to the rich comparison methods of the right arguments.

[serhiy-storchaka]

New changeset b9052a0 by Serhiy Storchaka in branch 'master':
bpo-31893: Fixed select.kqueue(). (bpo-4166)
b9052a0

[serhiy-storchaka]

New changeset f9a639b by Serhiy Storchaka (Miss Islington (bot)) in branch '3.6':
bpo-31893: Fixed select.kqueue(). (GH-4166) (bpo-4190)
f9a639b

[vstinner]

The commit b9052a0 broke compilation on FreeBSD:

building 'select' extension
cc -pthread -fPIC -Wno-unused-result -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -I./Include -I. -I/usr/local/include -I/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Include -I/usr/home/buildbot/python/3.x.koobs-freebsd10/build -c /usr/home/buildbot/python/3.x.koobs-freebsd10/build/Modules/selectmodule.c -o build/temp.freebsd-10.3-STABLE-amd64-3.7-pydebug/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Modules/selectmodule.o
/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Modules/selectmodule.c:1889:63: error: expected ';' at end of declaration
FILTER_FMT_UNIT FLAGS_FMT_UNIT FFLAGS_FMT_UNIT DATA_FMT_UNIT
^
;
1 error generated.
*** WARNING: renaming "_asyncio" since importing it failed: No module named 'select'

http://buildbot.python.org/all/#/builders/87/builds/95/steps/3/logs/stdio

[serhiy-storchaka]

New changeset 8cbf4e1 by Serhiy Storchaka in branch '2.7':
[2.7] bpo-31893: Fixed select.kqueue(). (GH-4166) (bpo-4193)
8cbf4e1

[vstinner]

Oh, the commit also broke the "x86 Tiger 3.x buildbot:

http://buildbot.python.org/all/#/builders/30/builds/93

======================================================================
FAIL: test_create_event (test.test_kqueue.TestKQueue)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/test/test_kqueue.py", line 71, in test_create_event
    self.assertEqual(ev.data, 5)
AssertionError: 25769803781 != 5

[serhiy-storchaka]

Oh, I forgot that Mac OS X also in the BSD family.

Thank you Victor for signaling errors.

[serhiy-storchaka]

New changeset 2298fad by Serhiy Storchaka in branch 'master':
bpo-31893: Fix errors in b9052a0. (bpo-4196)
2298fad

[serhiy-storchaka]

New changeset 84e252b by Serhiy Storchaka (Miss Islington (bot)) in branch '3.6':
bpo-31893: Fix errors in b9052a0. (GH-4196) (bpo-4201)
84e252b

[serhiy-storchaka]

New changeset e7531e5 by Serhiy Storchaka (Miss Islington (bot)) in branch '2.7':
bpo-31893: Fix errors in b9052a0. (GH-4196) (bpo-4202)
e7531e5

[serhiy-storchaka]

Tests on all buildbots are passed.