PEP 475: fnctl functions are not retried if interrupted by a signal (EINTR)

[akeskimo]

BPO	35189
Nosy	@vstinner, @aixtools, @miss-islington, @akeskimo, @nierob
PRs	bpo-35189: Retry fnctl calls on EINTR #10413 bpo-35189: Fix eintr_tester.py #10637 [3.7] bpo-35189: Retry fnctl calls on EINTR (GH-10413) #10678 [3.6] bpo-35189: Retry fnctl calls on EINTR (GH-10413) (GH-10678) #10685 bpo-35189, bpo-35316: Make test_eintr less strict #10782 [3.7] bpo-35189, bpo-35316: Make test_eintr less strict (GH-10782) #10784 [3.6] bpo-35189, bpo-35316: Make test_eintr less strict (GH-10782) #10785 bpo-35633: test_lockf() fails with "PermissionError: [Errno 13] Permission denied" on AIX #11424 bpo-35633: test_lockf() fails with "PermissionError: [Errno 13] Permission denied" on AIX #11424 bpo-35633: test_lockf() fails with "PermissionError: [Errno 13] Permission denied" on AIX #11424 bpo-35633: test_lockf() fails with "PermissionError: [Errno 13] Permission denied" on AIX #11424 [3.7] bpo-35633: test_lockf() fails with "PermissionError: [Errno 13] Permission denied" on AIX (GH-11424) #11858

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-11-23.18:01:48.601>
created_at = <Date 2018-11-08.12:45:30.367>
labels = ['3.8', '3.7', 'library', 'type-crash']
title = 'PEP 475: fnctl functions are not retried if interrupted by a signal (EINTR)'
updated_at = <Date 2019-02-14.18:41:41.577>
user = 'https://github.com/akeskimo'

bugs.python.org fields:

activity = <Date 2019-02-14.18:41:41.577>
actor = 'miss-islington'
assignee = 'none'
closed = True
closed_date = <Date 2018-11-23.18:01:48.601>
closer = 'vstinner'
components = ['Library (Lib)']
creation = <Date 2018-11-08.12:45:30.367>
creator = 'akeskimo'
dependencies = []
files = []
hgrepos = []
issue_num = 35189
keywords = ['patch']
message_count = 20.0
messages = ['329469', '329470', '329471', '329472', '329473', '330208', '330335', '330339', '330348', '330350', '330645', '330649', '330652', '332580', '332581', '332591', '332624', '333021', '335551', '335554']
nosy_count = 5.0
nosy_names = ['vstinner', 'Michael.Felt', 'miss-islington', 'akeskimo', 'nierob']
pr_nums = ['10413', '10637', '10678', '10685', '10782', '10784', '10785', '11424', '11424', '11424', '11424', '11858']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue35189'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

[akeskimo]

According to https://www.python.org/dev/peps/pep-0475/ the EINTR interruption should be retried automatically, but somehow it does not work and the exception is raised:

2018-11-05 05:21:35,257 ERROR:storage(23491): Remote storage operation failed (request: '{  'excludeSubModules': None,
   'storageLocation': 'qt/qtdatavis3d/68faa5b00f73096eb096c6acdfce76b052ca20b9/LinuxUbuntu_18_04x86_64LinuxQEMUarm64GCCqtci-linux-Ubuntu-18.04-x86_64-a6
c9f7Release/ac4280d182ec320eaf0e68efaeeeb6be14b9689f/test_1542834179',
   'type': 3}')
Traceback (most recent call last):
  File "src/storage.py", line 507, in handle
    self.handle_upload_artifact(message)
  File "src/storage.py", line 437, in handle_upload_artifact
    log.info("upload of %s to %s", uploadType, message.storageLocation)
  File "/usr/lib/python3.6/logging/__init__.py", line 1306, in info
    self._log(INFO, msg, args, **kwargs)
  File "/usr/lib/python3.6/logging/__init__.py", line 1442, in _log
    self.handle(record)
  File "/usr/lib/python3.6/logging/__init__.py", line 1452, in handle
    self.callHandlers(record)
  File "/usr/lib/python3.6/logging/__init__.py", line 1514, in callHandlers
    hdlr.handle(record)
  File "/usr/lib/python3.6/logging/__init__.py", line 861, in handle
    self.acquire()
  File "/home/vmbuilder/qt-ci/src/application.py", line 151, in acquire
    fcntl.lockf(self._lock_fd, fcntl.LOCK_EX)
InterruptedError: [Errno 4] Interrupted system call

[vstinner]

You're right, it should, but the fcntl module hasn't been patched. Are you interested to work or on patch, or do you want to me to find someone to do it?

[akeskimo]

My colleague has made a prospective fix:

nierob@3b76b88

[vstinner]

nierob@3b76b88

Oh, nice! Please rebase this change on the master branch and reuse "int async_err = 0;" pattern from Modules/posixmodule.c. You must not raise a new exception if PyErr_CheckSignals() raised an exception, something like:

    return (!async_err) ? posix_error() : NULL;

[nierob]

PR waits for CLA

[vstinner]

New changeset aac1f81 by Victor Stinner in branch 'master':
bpo-35189: Fix eintr_tester.py (GH-10637)
aac1f81

[vstinner]

New changeset b409ffa by Victor Stinner (nierob) in branch 'master':
bpo-35189: Retry fnctl calls on EINTR (GH-10413)
b409ffa

[vstinner]

New changeset 56742f1 by Victor Stinner in branch '3.7':
[3.7] bpo-35189: Retry fnctl calls on EINTR (GH-10413) (GH-10678)
56742f1

[vstinner]

New changeset eef813b by Victor Stinner in branch '3.6':
[3.7] bpo-35189: Retry fnctl calls on EINTR (GH-10413) (GH-10678) (GH-10685)
eef813b

[vstinner]

Thanks Aapo Samuli Keskimolo for the bug report and thanks Nierob for the fix!