BytesIO and StringIO values unavailable when closed

[vadmium]

BPO	23099
Nosy	@pitrou, @vadmium, @serhiy-storchaka
Files	bytesio_exported_reject_close.patch bytesio_exported_reject_close.v2.patch bytesio_exported_reject_close.v3.patch bytesio_exported_reject_close.v4.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2015-02-03.07:40:55.918>
created_at = <Date 2014-12-22.11:41:17.260>
labels = ['type-bug', 'expert-IO']
title = 'BytesIO and StringIO values unavailable when closed'
updated_at = <Date 2015-02-03.07:40:55.917>
user = 'https://github.com/vadmium'

bugs.python.org fields:

activity = <Date 2015-02-03.07:40:55.917>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2015-02-03.07:40:55.918>
closer = 'serhiy.storchaka'
components = ['IO']
creation = <Date 2014-12-22.11:41:17.260>
creator = 'martin.panter'
dependencies = []
files = ['37531', '37611', '37942', '37973']
hgrepos = []
issue_num = 23099
keywords = ['patch']
message_count = 12.0
messages = ['233016', '233018', '233023', '233036', '233506', '235108', '235138', '235159', '235222', '235229', '235304', '235317']
nosy_count = 4.0
nosy_names = ['pitrou', 'python-dev', 'martin.panter', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue23099'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5']

[vadmium]

IOBase.close() doc says file operations raise ValueError, but it is not obvious to me that reading back the “file” buffer is a file operation.

>>> with BytesIO() as b:
...     b.write(b"123")
... 
3
>>> b.getvalue()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: I/O operation on closed file.

Even worse, the memoryview gets corrupted on close():

>>> b = BytesIO(b"123")
>>> m = b.getbuffer()
>>> b.close()
>>> bytes(m)
b'\x98\x02>'

I also noticed that in the “io” implementation, writing to the file seems to be completely disallowed, even if it would not seem to change the size:

>>> b = BytesIO(b"123")
>>> m = b.getbuffer()
>>> b.write(b"x")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
BufferError: Existing exports of data: object cannot be re-sized

[serhiy-storchaka]

getvalue() doesn't work after close() for purpose. close() frees memory used by BytesIO.

>>> import io, sys
>>> bio = io.BytesIO()
>>> sys.getsizeof(bio)
52
>>> bio.write(b'x'*1000)
1000
>>> sys.getsizeof(bio)
1053
>>> bio.close()
>>> sys.getsizeof(bio)
52

Changing the behavior will cause regression.

The behavior of memoryview looks as a bug.

[pitrou]

Even worse, the memoryview gets corrupted on close():

The BytesIO object should probably reject closing when a buffer is exported.

writing to the file seems to be completely disallowed, even if it
would not seem to change the size:

An enhancement is probably possible there.

[serhiy-storchaka]

Here is a patch which rejects close() when a buffer is exported.

[vadmium]

Updated patch, to also document the BytesIO buffer is no longer available when closed. The StringIO documentation actually already says this, but I rarely use StringIO. :)

[serhiy-storchaka]

Why not just copy the StringIO documentation?

[vadmium]

I can live with the wording of StringIO, but personally prefer my v2 patch. I now understand that calling close() for Bytes and StringIO objects is intended to immediately free the memory buffer holding the file data (like deleting a file in Windows). So I think it would be better to document this as a property of the whole object, rather than a special exception for each of the getbuffer(), getvalue() non-file-API methods.

I’m happy to propose a similar wording for the StringIO class if you want to make them consistent.

[serhiy-storchaka]

Yes, it would be good to make the documentation of BytesIO and StringIO consistent.

[vadmium]

Here is an option that moves the documentation for discarding the buffer into the class description for both BytesIO and StringIO; what do you think? I would be happy enough with any of the last three patches, so I don’t want to hold this up forever :)

[serhiy-storchaka]

LGTM.

[python-dev]

New changeset e62d54128bd3 by Serhiy Storchaka in branch '3.4':
Issue bpo-23099: Closing io.BytesIO with exported buffer is rejected now to
https://hg.python.org/cpython/rev/e62d54128bd3

[python-dev]

New changeset b9d4c013b09a by Serhiy Storchaka in branch 'default':
Issue bpo-23099: Closing io.BytesIO with exported buffer is rejected now to
https://hg.python.org/cpython/rev/b9d4c013b09a