SSLSocket.send() returns 0 for non-blocking socket

[nikratio]

BPO	20951
Nosy	@pitrou, @giampaolo, @tiran, @bitdancer, @bdarnell
Files	issue20951.diff issue20951.diff issue20951.diff deprecation_patch.diff docpatch.diff issue20951_r2.diff issue20951_r3.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-04-29.08:06:41.244>
created_at = <Date 2014-03-16.21:16:31.275>
labels = ['type-feature', 'library']
title = 'SSLSocket.send() returns 0 for non-blocking socket'
updated_at = <Date 2014-05-01.12:09:37.606>
user = 'https://bugs.python.org/nikratio'

bugs.python.org fields:

activity = <Date 2014-05-01.12:09:37.606>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2014-04-29.08:06:41.244>
closer = 'pitrou'
components = ['Library (Lib)']
creation = <Date 2014-03-16.21:16:31.275>
creator = 'nikratio'
dependencies = []
files = ['34448', '34504', '34541', '34608', '34633', '35062', '35082']
hgrepos = []
issue_num = 20951
keywords = ['patch']
message_count = 31.0
messages = ['213759', '213761', '213764', '213766', '213774', '213776', '213778', '213779', '213780', '213782', '214042', '214121', '214316', '214772', '214847', '214848', '214876', '214877', '214878', '214889', '214931', '217322', '217483', '217484', '217485', '217489', '217492', '217567', '217583', '217673', '217689']
nosy_count = 8.0
nosy_names = ['janssen', 'pitrou', 'giampaolo.rodola', 'christian.heimes', 'r.david.murray', 'nikratio', 'python-dev', 'Ben.Darnell']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue20951'
versions = ['Python 3.5']

[nikratio]

When using non-blocking operation, the SSLSocket.send method returns 0 if no data can be sent at this point.

This is counterintuitive, because in the same situation (write to non-blocking socket that isn't ready for IO):

• A regular (non-SSL) socket raises BlockingIOError
• libc's send(2) does not return 0, but -EAGAIN or -EWOULDBLOCK.
• OpenSSL's ssl_write does not return 0, but returns an SSL_ERROR_WANT_WRITE error
• The ssl module's documentation describes the SSLWantWrite exception as "A subclass of SSLError raised by a non-blocking SSL socket when trying to read or write data, but more data needs to be sent on the underlying TCP transport before the request can be fulfilled."
• Consistent with that, trying to *read* from a non-blocking SSLSocket when no data is ready raises SSLWantRead, instead of returning zero.

This behavior also makes it more complicated to write code that works with both SSLSockets and regular sockets.

Since the current behavior undocumented at best (and contradicting the documentation at worst), can we change this in Python 3.5?

[nikratio]

This is actually seems to be not just an inconvience, but a real bug: since SSLSocket.sendall() uses SSLSocket.send() internally, the former method will busy-loop when called on a non-blocking socket.

Note also that the .sendto and .write methods already behave consistently and raise SSLWantWrite. It seems it's really just the send() method that is the lone outlier.

The attached patch changes ssl.send to raise SSLWantWrite instead of returning zero. The full testsuite still runs fine. I'm a bit sceptical though, because the code looks as if send() was deliberately written to catch the SSLWantWrite exception and return zero instead.. Can anyone familiar with the code comment on this?

[bitdancer]

A little hg sleuthing (which I assume you did but I'll record for the record) reveals that this was introduced by Bill Jansen in changeset 8a281bfc058d. Following the bugs mentioned in the checkin message, it looks like it *might* have been related to bpo-1251, but there really isn't enough information in the issues or the checkin to tell for sure. It certainly sounds like the problems mentioned in that issue may be relevant, though (the disconnection between the unecrypted data send and what actually gets placed on the wire and when).

I see you already added Bill Jansen to nosy, so that's probably the best bet for getting an answer, if we are lucky and he both responds and remembers :)

[pitrou]

It's probably too late to change this, unfortunately. There are non-blocking frameworks and libraries out there relying on the current behaviour.

As for sendall(), it doesn't really make sense on a non-blocking socket anyway.

[nikratio]

Antoine, do you know that there are frameworks out there using this, or is that a guess? asyncio, for example, seems to expect an SSLWantWrite exception as well. (it also works with a zero return, but it's not clear from the code if that's by design or by a chance).

[pitrou]

Antoine, do you know that there are frameworks out there using this,
or is that a guess?

It's just a guess.

[nikratio]

Twisted does not seem to rely on it either (there's no mention of SSLWant* in the source at all, and without that, you can't possibly have support for non-blocking ssl sockets).

[nikratio]

gevent is calling _sslobject.write() directly, so it would not be affected by any change.

[nikratio]

Tornado uses SSLSocket.send(), and it looks as if a SSLWantWrite exception is not caught but would propagate, so this would probably break.

[nikratio]

More info on twisted: it uses PyOpenSSL rather than the stdlib ssl module, so it's not affected at all.

[nikratio]

Since this behavior cannot be changed without breaking third-party libraries (why did they work around this rather than reporting a bug?), I'd suggest to document the current behavior and allow programs to opt-in to getting exceptions.

I've attached a patch to that end. Feedback would be appreciated.