Add method send_io, recv_io to the socket module.

[tokibito]

BPO	28724
Nosy	@vstinner, @tiran, @methane, @JulienPalard, @tokibito, @miss-islington, @nanjekyejoannah, @gousaiyang, @hauntsaninja
PRs	bpo-28724: Add methods send_fds, recv_fds to the socket module #12889 bpo-28724: Move socket.send_fds and socket.recv_fds documentation to the "Other functions" section #22608 [3.9] bpo-28724: Doc: Move socket.send_fds and socket.recv_fds docs to right section (GH-22608) #25517
Files	socket_send_io_recv_io.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-04-22.04:35:33.652>
created_at = <Date 2016-11-17.15:05:33.604>
labels = ['type-feature', 'library', '3.9']
title = 'Add method send_io, recv_io to the socket module.'
updated_at = <Date 2021-04-22.04:35:33.651>
user = 'https://github.com/tokibito'

bugs.python.org fields:

activity = <Date 2021-04-22.04:35:33.651>
actor = 'methane'
assignee = 'none'
closed = True
closed_date = <Date 2021-04-22.04:35:33.652>
closer = 'methane'
components = ['Library (Lib)']
creation = <Date 2016-11-17.15:05:33.604>
creator = 'tokibito'
dependencies = []
files = ['45519']
hgrepos = []
issue_num = 28724
keywords = ['patch']
message_count = 13.0
messages = ['281041', '281065', '281089', '282348', '282415', '282488', '282508', '346379', '351986', '351988', '376132', '391573', '391575']
nosy_count = 9.0
nosy_names = ['vstinner', 'christian.heimes', 'methane', 'mdk', 'tokibito', 'miss-islington', 'nanjekyejoannah', 'gousaiyang', 'hauntsaninja']
pr_nums = ['12889', '22608', '25517']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue28724'
versions = ['Python 3.9']

[tokibito]

This patch makes it easy to pass file descriptor in using AF_UNIX.

Ruby language libraries have such methods.

see slso:

• https://docs.ruby-lang.org/en/2.3.0/UNIXSocket.html#method-i-send_io

[JulienPalard]

Hi, thanks for your contribution!

Documentation give examples implementation of your methods:

• https://docs.python.org/3/library/socket.html#socket.socket.sendmsg
• https://docs.python.org/3/library/socket.html#socket.socket.recvmsg

and from here, some remarks:

• Why allowing people to mix fds and file objects? I'm not a fan of this: I prefer the clarity of allowing only file descriptors. Have you seen such methods (allowing both fd and file obj) in the stdlib?
• The documented implementation for recv_io checks for integer truncation, there may be a good reason, should'n you do the same?
• The documented implementation allows to pass a message, shouldn't you at least allow for an optional message to be passed?

Adding those methods may make sense if people are copying send_fds/recv_fds in their code, but I found only two copy/paste in github, so I think the best is to put your code as a module on pypi and see, from here, if it gets popular?

You should also write a few tests, and comply to the PEP-8 (your two methods should probably be separated by a newline).

Bests

[methane]

FYI, multiprocessing.reduction module has send_fds and recv_fds.

[JulienPalard]

Unless somebody don't think so, I think this should go as a pypi module before going to the socket module, so this issue should probably be closed.

[tiran]

Good idea! Initially I planned to add the functions and some other helpers around AF_UNIX to Python 3.6. You can directly copy the example functions from the documentation.

https://docs.python.org/3/library/socket.html#socket.socket.sendmsg
https://docs.python.org/3/library/socket.html#socket.socket.recvmsg

Please keep the names send_fds() and recv_fds(), and pass the message like in the examples. Some protocols use the payload to signal the count of fds. Your patch is also lacking unit tests and documentation update.

[tokibito]

Thanks for reviews and comments. I will work on the weekend.

[tiran]

Take your time, feature freeze for 3.7 is in about 18 months from now.

[vstinner]

See also bpo-37385: "test_multiprocessing fails on AMD64 FreeBSD CURRENT Shared 2.7".

[vstinner]

New changeset 8d120f7 by Victor Stinner (Joannah Nanjekye) in branch 'master':
bpo-28724: Add methods send_fds and recv_fds to the socket module (GH-12889)
8d120f7

[vstinner]

Thanks Shinya Okano for the original patch.

Well done Joannah! Thanks for your tenacity :-) This PR has 100 comments and 27 commits which shows the complexity of the feature.

Honestly, I'm not 100% happy with current documentation, but I chose to merge the PR anyway. Please leave this issue open until the documentation is completed to mention corner cases like partial send (similar to sock.send vs sock.sendall).

[hauntsaninja]

I was looking at adding stubs for these to typeshed. Is it intentional that we ignore the flags and address arguments in the implementation?

[methane]

New changeset 660592f by Saiyang Gou in branch 'master':
bpo-28724: Doc: Move socket.send_fds and socket.recv_fds docs to right section (GH-22608)
660592f