Speedup readexactly

[socketpair]

See #394

[socketpair]

I still not tested performance. Please be patient, or help me with that :)

[methane]

while len(self._buffer) < n:

[socketpair]

fixed

[socketpair]

#394 should be fixed with that. Will benchmark after some time.

[1st1]

@socketpair I can merge this PR in 3.6 today/tomorrow if you can show that it improves performance.

[socketpair]

In non-cpython implementations, destructing of memoryview may be postponed, so cutting buffer may fail. In order to fix that, we should call memoryview_obj.release()...

Not sure if it is actual.

[methane]

Ah, you're right, I think.
I wonder if there is more simple and safe API.

[socketpair]

To overcome that, it is require to make two separate variables: i.e.

memview1 = memoryview(self._buffer)
memview2 = memview1[:n]
data = bytes(memview2)
memview2.release()
memview1.release()

So, I decide to eliminate memoryview...Yes it should be faster, but....

[gvanrossum]

I think you should watch your language.

[socketpair]

Sorry, I have fixed comment.

[socketpair]

I have benchmarked that.
8.72 on master branch
6.65 on this branch.

program:

#!/usr/bin/env python3.5

import asyncio
import os
import time

async def action(rs, chunksize):
    try:
        while True:
            await rs.readexactly(chunksize)
    except asyncio.IncompleteReadError:
        pass

async def writer(ws, bufsize):
    ws.write(b'x' * bufsize)
    await ws.drain()
    ws.close()

async def amain():
    bufsize = 30 * 1024 * 1024
    chunksize = 16

    handler = lambda rs, ws: writer(ws, bufsize)
    server = await asyncio.start_server(handler, '127.0.0.1', 0)
    addr = server.sockets[0].getsockname()
    (rs, ws) = await asyncio.open_connection(*addr)
    server.close()
    await server.wait_closed()
    # now writing part is running "in background". TODO: wait it completion somehow

    await asyncio.sleep(3)

    a = time.monotonic()
    await action(rs, chunksize)
    b = time.monotonic()
    ws.close()
    print('chunk size={}: {:.2f} seconds.'.format(
        chunksize,
        b - a,
    ))


loop = asyncio.get_event_loop()
loop.run_until_complete(amain())

[socketpair]

Adding memoryview slowers down (!) so, it is good not to use memoryview before copying in that place.

[socketpair]

@gvanrossum @methane @1st1

Any feedback ?

[socketpair]

Really frequent case for protocols like this: http://www.postfix.org/socketmap_table.5.html where we expect a string with known length, data after which is not expected.

(Did not benchmarked if this really speedup)

[methane]

IMO, I want to add helper C function for "slicing bytes from bytearray".
And I hope it will be part of Python 3.7, (e.g. bytes.frombuffer(buff, start=0, end=None)).

[methane]

lgtm

[socketpair]

@1st1 @gvanrossum ping

[1st1]

I'll commit this later today.

[1st1]

Committed by hand in b0e7438. Thanks!