zipfile.ZipFile.close() doesn't respect allowZip64

[bgilbert]

BPO	21866
Nosy	@serhiy-storchaka
Files	zipfile64_additional_checks.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2014-09-23.20:31:16.430>
created_at = <Date 2014-06-25.05:56:09.484>
labels = ['type-bug', 'library']
title = "zipfile.ZipFile.close() doesn't respect allowZip64"
updated_at = <Date 2014-09-23.20:31:16.429>
user = 'https://bugs.python.org/bgilbert'

bugs.python.org fields:

activity = <Date 2014-09-23.20:31:16.429>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2014-09-23.20:31:16.430>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2014-06-25.05:56:09.484>
creator = 'bgilbert'
dependencies = []
files = ['36548']
hgrepos = []
issue_num = 21866
keywords = ['patch']
message_count = 6.0
messages = ['221525', '221543', '222476', '222489', '226420', '227372']
nosy_count = 3.0
nosy_names = ['python-dev', 'bgilbert', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue21866'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5']

[bgilbert]

The ZipFile documentation says:

If allowZip64 is True (the default) zipfile will create ZIP files that
use the ZIP64 extensions when the zipfile is larger than 2 GiB. If it
is false zipfile will raise an exception when the ZIP file would
require ZIP64 extensions.

ZipFile.close() will write ZIP64 central directory records if e.g. a member's local file header starts at an offset > 2 GB, or if there are more than 65535 files in the archive. It will do this even if allowZip64 is False, whereas the documentation implies that it should raise an exception in that case.

[serhiy-storchaka]

Do you want to provide a patch?

[serhiy-storchaka]

An offset is checked in write() and writestr() methods and an exception is raised if an offset is >= 2 GiB and ZIP64 extension is not allowed. I don't see a possibility for a bug.

[bgilbert]

Here are the cases where close() will generate a ZIP64 archive and an exception will never be raised:

1. There are more than 65535 files in the archive.

2. The start of the central directory is at > 2 GB.

3. The central directory size is > 2 GB.

#1 could be checked from write/writestr. #2 could possibly be checked from write/writestr by looking at the file offset after writing the compressed data. #3 cannot be checked until close, but I'm not sure if it can ever occur without triggering one of the other checks first.

[serhiy-storchaka]

Thank you Benjamin for explanation.

Here is a patch which adds all three required checks in the close() method and adds a check for files count in write methods. Implementing other checks in write methods is too laborious.

As far as test_zipfile64 is disabled, for manual testing you need temporary comment out the "support.requires()" statement and run only selected tests:

./python -m test.regrtest -v -m OtherTests test_zipfile64

[python-dev]

New changeset 8a010ca89094 by Serhiy Storchaka in branch '2.7':
Issue bpo-21866: ZipFile.close() no longer writes ZIP64 central directory
https://hg.python.org/cpython/rev/8a010ca89094

New changeset 8f25d118ce38 by Serhiy Storchaka in branch '3.4':
Issue bpo-21866: ZipFile.close() no longer writes ZIP64 central directory
https://hg.python.org/cpython/rev/8f25d118ce38

New changeset d361d2176121 by Serhiy Storchaka in branch 'default':
Issue bpo-21866: ZipFile.close() no longer writes ZIP64 central directory
https://hg.python.org/cpython/rev/d361d2176121