Add socket.bind_socket() convenience function

[giampaolo]

BPO	17561
Nosy	@loewis, @gpshead, @jaraco, @nirs, @pitrou, @vstinner, @giampaolo, @bitdancer, @asvetlov, @berkerpeksag, @vadmium, @jleedev, @csabella
PRs	bpo-17561: add socket.create_server_sock() and socket.has_dual_stack() #11215 bpo-35934: Add socket.create_server() utility function #11784 BPO-17561: set create_server backlog default to None #12735
Files	socket.patch: adds create_server_sock() socket2.patch: disable IPV6_V6ONLY, adds has_dual_stack()

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/giampaolo'
closed_at = <Date 2019-04-09.10:34:52.387>
created_at = <Date 2013-03-27.16:48:29.927>
labels = ['3.8', 'type-feature', 'library']
title = 'Add socket.bind_socket() convenience function'
updated_at = <Date 2019-04-09.17:53:09.958>
user = 'https://github.com/giampaolo'

bugs.python.org fields:

activity = <Date 2019-04-09.17:53:09.958>
actor = 'vstinner'
assignee = 'giampaolo.rodola'
closed = True
closed_date = <Date 2019-04-09.10:34:52.387>
closer = 'giampaolo.rodola'
components = ['Library (Lib)']
creation = <Date 2013-03-27.16:48:29.927>
creator = 'giampaolo.rodola'
dependencies = []
files = ['29596', '29600']
hgrepos = []
issue_num = 17561
keywords = ['patch', 'needs review']
message_count = 29.0
messages = ['185348', '185362', '185369', '185370', '185372', '185374', '185443', '185447', '185456', '185483', '185486', '185487', '185489', '185497', '185499', '185765', '185849', '185919', '327719', '331943', '331944', '335021', '335036', '335285', '339678', '339687', '339694', '339723', '339794']
nosy_count = 20.0
nosy_names = ['loewis', 'gregory.p.smith', 'jaraco', 'nirs', 'pitrou', 'vstinner', 'giampaolo.rodola', 'josiah.carlson', 'r.david.murray', 'asvetlov', 'neologix', 'berker.peksag', 'martin.panter', 'jpokorny', 'jleedev', 'dazhaoyu', 'andreasr', 'Carlos.Ralli', 'Paul Marks', 'cheryl.sabella']
pr_nums = ['11215', '11784', '12735']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue17561'
versions = ['Python 3.8']

[giampaolo]

Here's a function similar to socket.create_connection() which addresses all the repetitive tasks needed to order to create an IPv4/IPv6 agnostic server socket.

[neologix]

I think that's a good idea.

However, there's a problem with the implementation: if one passes "" or None as address on a dual-stack node, the resulting socket will be either IPv4 bound to INADDR_ANY, or IPv6 bound to IN6ADDR_ANY, whereas one would expect to be bound both in IPv4 and IPv6. In the later case, some platforms (like Linux) use IPv4-mapped addresses by default, some others (e.g. some versions of FreeBSD) don't. So, depending on IPV6_V6ONLY setting, binding to IPV6 any won't accept IPv4 connections. Also, some platforms don't support mapped addresses at all, so the only portable solution is to bind both to 0.0.0.0 and ::, whith two different sockets, and use select() to accept both.

So it would maybe make sense to expose a ServerSocket class, with an accept method which would do the right thing (as an added bonus, it could expose set_reuse_addr(bool), since SO_REUSEADDR have subtle semantic differences between Unix and Windows).

[giampaolo]

What you say is right but whether the kernel supports an hybrid IPv4/6 stack or not there's not much we can do about it anyway.
Exactly what are you suggesting with the ServerSocket class you mentioned? What do you expect it to do?
Note that platforms supporting the dual-stack are already supported. This:

>> socket.create_server_socket(("::", 0))

...on Linux will create a socket which is reachable both as "::1" and "127.0.0.1".
So if I'm not mistaken the alias for specifying "listen on all interfaces for both IPv4 and IPv6 addresses" would be "::" instead of "" / None.
We can document that.

[giampaolo]

Side note: this is how in pyftpdlib I determine whether a platform supports the dual stack:

def support_hybrid_ip_v4_v6():
    # Note: IPPROTO_IPV6 constant is broken on Windows, see:
    # http://bugs.python.org/issue6926
    sock = None
    try:
        if not socket.has_ipv6:
            return False
        sock = socket.socket(socket.AF_INET6)
        return not sock.getsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY)
    except (socket.error, AttributeError):
        return False
    finally:
        if sock is not None:
            sock.close()

[gvanrossum]

Tulip has something similar. Someone should compare the two and make sure they are equivalent or similar.

[neologix]

What you say is right but whether the kernel supports an hybrid IPv4/6 stack
or not there's not much we can do about it anyway.
Exactly what are you suggesting with the ServerSocket class you mentioned?
What do you expect it to do?

There's a confusion.
Dual-stack merely means that the host supports both IPv4 and IPv6 natively.
The IPV6_V6ONLY tunable is just a way to enable or not IPv4-mapped
addresses (i.e. ::ffff::<IPv4 address>) so that an IPv4 client can
connect to this socket. It can be set system-wide, or on a socket
basis.

Note that platforms supporting the dual-stack are already supported. This:

>>> socket.create_server_socket(("::", 0))

...on Linux will create a socket which is reachable both as "::1" and
"127.0.0.1".

Try the same thing after:
# echo 1 > /proc/sys/net/ipv6/bindv6only

It won't accept IPv4 connections anymore.

And that's the default on many (most) platforms, e.g. FreeBSD and
OpenBSD (I think it's also true for Windows).

So the bottom line is that with the current code, on some - most -
platforms, we'll only accept IPv6 connections (and since you iterate
over getaddrinfo() in an unspecified order you may very well bind to
IPv4 only first, in which case you'll only accept IPv4 clients).

The proper way to procedd, on platforms which don't support unsetting
IPV6_V6ONLY, is to use two sockets, one in IPv4, and one IPv6, and use
select() to accept connections.

This would propably belong to an overriden accept() method in a
ServerSocket class, since it's far from trivial.

[giampaolo]

Thanks for clarifying, I have a better understanding of the problem now.
Providing a custom class instantiating two sockets looks like a dead end to me though. To say one, what is getsockname() supposed to return? Same for detach(), fileno(), 'family' and probably others I can't think of right now.

[neologix]

Providing a custom class instantiating two sockets looks like a dead end to me though. To say one, what is getsockname() supposed to return? Same for detach(), fileno(), 'family' and probably others I can't think of right now.

Indeed.

So we might opt for a best-effort approach: try disabling IPV6_V6ONLY
on the socket if it's set: that way it should work on most platforms.
And add a note to the documentation. IIRC that's what Java does.

[giampaolo]

Agreed. Then it probably makes sense to expose also a socket.has_dual_stack() function so that the user can pre-emptively decide whether using a custom class.
Updated draft patch is in attachment.

[giampaolo]

I managed to write a container class which listens on multiples addresses and uses select/poll on accept() as suggested by Charles.
FWICT it seems to work pretty well and supports non-blocking sockets and timeouts (tested on Linux, Windows and OSX).

It is available as a recipe here:
http://code.activestate.com/recipes/578504

IMO has_dual_stack() and create_server_sock() functions can already be included as-is (will adapt the recipe in order to get rid of Python 2.X support and submit another patch).

As for including also MultipleSocketsListener that's debatable.
It can either be added or linked/mentioned it in the doc.

[gvanrossum]

Perhaps you can contribute something like this to Tulip? We've got code to run a server that can handle IPv4 and IPv6, but we currently don't have something that just creates a separate socket for each address family. Our UDP and TCP paths are also quite different.

[giampaolo]

Yep, no prob. It would also be a good chance to test it in a real-world app. Where should I look?