Implement asyncio.Task in C

[1st1]

BPO	28544
Nosy	@gvanrossum, @brettcannon, @vstinner, @ned-deily, @asvetlov, @methane, @elprans, @1st1
Files	ctask.patch ctask2.patch ctask3.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/1st1'
closed_at = <Date 2016-10-29.02:17:28.753>
created_at = <Date 2016-10-27.16:33:11.278>
labels = ['expert-asyncio', '3.7', 'performance']
title = 'Implement asyncio.Task in C'
updated_at = <Date 2016-10-31.17:00:26.349>
user = 'https://github.com/1st1'

bugs.python.org fields:

activity = <Date 2016-10-31.17:00:26.349>
actor = 'brett.cannon'
assignee = 'yselivanov'
closed = True
closed_date = <Date 2016-10-29.02:17:28.753>
closer = 'yselivanov'
components = ['asyncio']
creation = <Date 2016-10-27.16:33:11.278>
creator = 'yselivanov'
dependencies = []
files = ['45242', '45245', '45248']
hgrepos = []
issue_num = 28544
keywords = ['patch']
message_count = 25.0
messages = ['279546', '279549', '279553', '279554', '279560', '279564', '279568', '279569', '279570', '279577', '279578', '279579', '279580', '279613', '279614', '279618', '279629', '279630', '279632', '279635', '279636', '279638', '279645', '279657', '279805']
nosy_count = 9.0
nosy_names = ['gvanrossum', 'brett.cannon', 'vstinner', 'ned.deily', 'asvetlov', 'methane', 'Elvis.Pranskevichus', 'python-dev', 'yselivanov']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue28544'
versions = ['Python 3.6', 'Python 3.7']

[1st1]

The attached patch implements asyncio.Task in C. Besides that, it also implements Argument Clinic for C Future.

Performance improvement on a simple echo server implementation using asyncio.streams:

Python Future & Task | C Future & Py Task | C Future & C Task
23K req/s | 26K | 30K
| ~10-15% boost | ~15%

Both Task and Future implemented in C, make asyncio programs up to 25-30% faster.

The patch is 100% backwards compatible. I modified asyncio tests to cross test Tasks and Futures implementations, i.e. to run Task+Future, Task+CFuture, CTask+Future, CTask+CFuture tests. No refleaks or other bugs. All uvloop functional tests pass without any problem.

Ned, Guido, are you OK if I merge this in 3.6 before beta 3? I'm confident that the patch is stable and even if something comes up we have time to fix it or even retract the patch. The performance boost is very impressive, and I can also make uvloop simpler.

[1st1]

Also, with this patch uvloop becomes ~3-5% faster too.

[methane]

Wow! Great Job!
Yury, would you like to merge this before 3.6b3?
I'll look this as soon as possible.

(nit fix) Some modules doesn't sort imports in lexicography.

[1st1]

Yury, would you like to merge this before 3.6b3?

Yes!

I'll look this as soon as possible.

Thanks a lot!

[ned-deily]

If it's OK with Guido, it's OK with me for 360b3. (That's Monday.)

[asvetlov]

Very impressive.
I've left a couple comments in rietveld though.

[gvanrossum]

I don't want to be the decider here. I don't have time to review the
code. I trust you all to do the right thing.

[ned-deily]

I also trust Yury to do the right thing.

[1st1]

Guido, Ned, thanks! Andrew already glanced through the code, let's see what Inada-san says.

I'm uploading an updated patch addressing Andrew's review.

[1st1]

Uploading a new patch: sorted imports; added a unittest that exceptions in loop.call_soon aren't breaking Task.__init__.

[methane]

Why isfuture() is moved, and asyncio.coroutine uses
base_futures.isfuture() instead of futures.isfuture()?