_pyio module broken on Cygwin / setmode not usable

[embray]

BPO	28459
Nosy	@embray, @zware, @ma8ma
PRs	bpo-28459: Fix _pyio on Cygwin where the msvcrt module is not built #14013
Superseder	bpo-45537: Cygwin is unsupported - close all open issues and list them here.
Files	cygwin-pyio-setmode.patch: included other parts

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-10-20.12:10:50.164>
created_at = <Date 2016-10-17.10:58:12.374>
labels = ['3.7', 'library', 'expert-IO', 'type-crash']
title = '_pyio module broken on Cygwin / setmode not usable'
updated_at = <Date 2021-10-21.09:57:21.024>
user = 'https://github.com/embray'

bugs.python.org fields:

activity = <Date 2021-10-21.09:57:21.024>
actor = 'iritkatriel'
assignee = 'none'
closed = True
closed_date = <Date 2021-10-20.12:10:50.164>
closer = 'iritkatriel'
components = ['Library (Lib)', 'IO']
creation = <Date 2016-10-17.10:58:12.374>
creator = 'erik.bray'
dependencies = []
files = ['45163']
hgrepos = []
issue_num = 28459
keywords = ['patch']
message_count = 5.0
messages = ['278802', '279086', '280203', '308174', '308630']
nosy_count = 3.0
nosy_names = ['erik.bray', 'zach.ware', 'masamoto']
pr_nums = ['14013']
priority = 'normal'
resolution = 'wont fix'
stage = 'resolved'
status = 'closed'
superseder = '45537'
type = 'crash'
url = 'https://bugs.python.org/issue28459'
versions = ['Python 3.6', 'Python 3.7']

[embray]

Since the patch to bpo-24881 was merged the _pyio module has been non-importable on Cygwin, due to an attempt to import from the msvcrt module.

However, the msvcrt module is not built on Cygwin (Cygwin does not use MSVCRT). Cygwin's libc does, however, have _setmode.

The problem this is trying to solve is to call _setmode (https://msdn.microsoft.com/en-us/library/tw4k6df8.aspx) on file descriptors to ensure that they are O_BINARY instead of O_TEXT (a distinction that needs to made on Windows).

Fortunately, on Cygwin, it is fairly rare that a file descriptor will have mode O_TEXT, but it it is possible. See https://cygwin.com/faq/faq.html#faq.programming.msvcrt-and-cygwin

This could be tricky to solve though. Removing setmode() call entirely works for me as far as the test suite is concerned. But it leaves _pyio slightly incongruous with the C implementation in this one small aspect, and it *is* a bug.

I would propose for Python 3.7 adding an os.setmode() function. On Windows this could be simply an alias for msvcrt.setmode() (or vice-versa). But because setmode() is available also in Cygwin (and technically some other platforms too, though none that are currently supported by Python) it would be a good candidate for inclusion in the os module I think (for those platforms that have it).

[ma8ma]

I agree to use setmode() at _pyio module for Cygwin. However, I have two reasons that I disagree to the implementation of os.setmode().
First, FreeBSD has another setmode() that operate file permission bits [1]. Therefore the implementation of os.setmode() will be confused to those users.
Second, msvcrt.setmode() isn't used almost in standard library, the function is just used by _pyio module and subprocess test. Thus I think there aren't much need that implements setmode() to os module.

Easy way for a solution of import error on Cygwin, setmode() is implemented on _pyio using ctypes. then Cygwin CPython forget setmode().
Otherwise it's hard way, For avoiding confliction to other platforms, I'd propose to create the cygwin module. I thought a solution implementing imitation msvcrt module, but it is unreasonable because Cygwin doesn't have other Windows API.

I tried written the patch of easy way that implements setmode() into _pyio. This patch is included parts to succeed import _pyio.

[1] https://www.freebsd.org/cgi/man.cgi?query=setmode&sektion=3

[embray]

Hi Masayuki, thanks for the response (I thought I replied to this earlier but maybe I only imagined it--I've been on vacation).

I agree with just about everything you write here.

I'm aware of the FreeBSD setmode(), but I figured os.setmode() could do different things on different platforms as applicable. But that would also be very confusing.

Your patch using ctypes looks fine to me--I considered doing the same, but what I'm not sure is if it's kosher to use ctypes here, given that it's techically an optional module. Since _pyio is, as I understand it, mainly used for testing it's probably fine? But I think we'll need some core maintainers' comments here...

Implementing a cygwin module would be really nice, actually, even if it isn't included in the stdlib. There are a few other cygwin-specific APIs that it is useful to have wrappers around: https://cygwin.com/cygwin-api/cygwin-functions.html But that would be outside the scope of fixing this issue.

[ma8ma]

FYI, cygwin-pyio-setmode.patch includes two extra parts for running on Cygwin. First, fix import error for ctypes (unopened issue). Second, fix building _ctypes module (bpo-4032 - PR 4153).

[embray]

Right, the current patch is really combining several issues. I'll make a PR from it that just fixes the _pyio issue. I'll also open an issue for fixing the ctypes bug (this is a patch I've had in my cygwin branch for ages but just never got around to making a report for...)