TypeError in event loop finalizer, new in Python 3.4.3

[oconnor663]

BPO	23548
Nosy	@gvanrossum, @vstinner, @jab, @1st1, @oconnor663, @wjwwood

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-03-05.20:54:53.677>
created_at = <Date 2015-02-28.16:13:01.454>
labels = ['type-crash', 'expert-asyncio']
title = 'TypeError in event loop finalizer, new in Python 3.4.3'
updated_at = <Date 2018-06-20.02:08:15.216>
user = 'https://github.com/oconnor663'

bugs.python.org fields:

activity = <Date 2018-06-20.02:08:15.216>
actor = 'wjwwood'
assignee = 'none'
closed = True
closed_date = <Date 2015-03-05.20:54:53.677>
closer = 'vstinner'
components = ['asyncio']
creation = <Date 2015-02-28.16:13:01.454>
creator = 'oconnor663'
dependencies = []
files = []
hgrepos = []
issue_num = 23548
keywords = []
message_count = 18.0
messages = ['236892', '236894', '236904', '236923', '236960', '237242', '237258', '237259', '237274', '237286', '239499', '239500', '239524', '240248', '248274', '248286', '259205', '320010']
nosy_count = 8.0
nosy_names = ['gvanrossum', 'vstinner', 'jab', 'yselivanov', 'oconnor663', 'wjwwood', 'chetan', '\xe5\xb1\xb1\xe6\x9c\xac\xe6\xb3\xb0\xe5\xae\x87']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = None
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue23548'
versions = ['Python 3.4']

[oconnor663]

This toy program:

import asyncio
@asyncio.coroutine
def main():
    p = yield from asyncio.create_subprocess_shell('echo hi')
    yield from p.wait()
asyncio.get_event_loop().run_until_complete(main())

Produces this output on Arch Linux under Python 3.4.3 (but not 3.4.2):

hi
Exception ignored in: <bound method _UnixSelectorEventLoop.__del__ of <_UnixSelectorEventLoop running=False closed=True debug=False>>
Traceback (most recent call last):
  File "/home/jacko/Downloads/Python-3.4.3/Lib/asyncio/base_events.py", line 361, in __del__
  File "/home/jacko/Downloads/Python-3.4.3/Lib/asyncio/unix_events.py", line 57, in close
  File "/home/jacko/Downloads/Python-3.4.3/Lib/asyncio/unix_events.py", line 138, in remove_signal_handler
TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object

[gvanrossum]

Does it also have that error if you add asyncio.get_event_loop().close() to the end of the program?

Can you figure out what the value of sig and handler are in the traceback?

My gut feeling tells me this is due to signal.default_int_handler being None at that point in the program tear-down sequence. (Which is why I recommend closing the loop.) The tear-down sequence at best has a partial order, which is why this may come and go depending on random variables like which Python version you use.

[oconnor663]

close() fixes it; thanks for the workaround! When I throw a print statement inside remove_signal_handler, it says that sig is 17 and handler is 0. 17 looks to be SIGCHLD, presumably from the little echo subprocess exiting in this example.

[gvanrossum]

So this is still strange. When you do this, does it give the same exception?

>> import signal
>> signal.signal(signal.SIGCHLD, signal.SIG_DFL)

[vstinner]

I'm aware of the issue but I would prefer to not fix it.

The first problem is that you didn't close the event loop. I modified
recently the doc to explain at the begining that you should develop in the
debug mode. In this mode, you will see a warning if you don't close
explicitly the event loop.

The signal issue is that the event loop is closed very late during Python
finalization, and attributes of the signal module are already cleared.

Are you ok to not workaround this specific issue?

Closing an event loop implicitly late can lead to other various bugs.

[oconnor663]

@gvanrossum, the last two lines you suggested don't give any error, as expected. Not sure why we're getting that error message in the toy example.

@Haypo, closing the event loop explicitly works fine for me. But since you mentioned it, I don't see any warning when I leave out closing the event loop and turn on PYTHONASYNCIODEBUG=1. What should I be seeing?

[vstinner]

Did you read the latest doc? You should also use the -Wd command line
option to see resource warnings.

[vstinner]

Debug mode:
https://docs.python.org/dev/library/asyncio-dev.html#asyncio-debug-mode

[oconnor663]

Got it, thanks for the heads up.

[vstinner]

I close the issue as wont fix. I may workaround the bug during Python finalization if more users report this issue.

[jab]

Quoting Victor Stinner:

I may workaround the bug during Python finalization if more users report this issue.

Read the above so reporting I'm hitting this too fwiw.

Thanks for the great work on asyncio.

[jab]

Not sure if it's related / helpful but just in case, since upgrading from 3.4.2 to 3.4.3, I'm now seeing this printed to stderr sometimes when my program exits:

Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: