python -mzipfile fails to unzip files with folders created by zip

[anntzer]

BPO	22201
Nosy	@serhiy-storchaka, @anntzer
Files	fix_zipfile_empty_dir.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2014-08-17.12:28:33.139>
created_at = <Date 2014-08-14.23:27:42.299>
labels = ['type-bug', 'library']
title = 'python -mzipfile fails to unzip files with folders created by zip'
updated_at = <Date 2014-08-17.12:28:33.137>
user = 'https://github.com/anntzer'

bugs.python.org fields:

activity = <Date 2014-08-17.12:28:33.137>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2014-08-17.12:28:33.139>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2014-08-14.23:27:42.299>
creator = 'Antony.Lee'
dependencies = []
files = ['36389']
hgrepos = []
issue_num = 22201
keywords = ['patch']
message_count = 4.0
messages = ['225325', '225410', '225438', '225439']
nosy_count = 4.0
nosy_names = ['python-dev', 'serhiy.storchaka', 'Antony.Lee', 'ryantimwilson']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue22201'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5']

[anntzer]

With Python 3.4.1:

$ mkdir foo; zip -r foo{,}; python -mzipfile -e foo.zip dest
  adding: foo/ (stored 0%)
Traceback (most recent call last):
  File "/usr/lib/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.4/zipfile.py", line 1799, in <module>
    main()
  File "/usr/lib/python3.4/zipfile.py", line 1777, in main
    with open(tgt, 'wb') as fp:
IsADirectoryError: [Errno 21] Is a directory: 'dest/foo/'

Note that zipfile is actually able to unzip the file:

$ python -c 'from zipfile import *; ZipFile("foo.zip").extractall("dest")'

works fine.

If the zip file is created by "python -mzipfile -c foo.zip foo" then there is no problem. Likewise, if there are no folders in the zip file (but just a collection of files) then there is no problem.

[ryantimwilson]

The reason behind this was that zipfile.py currently handles empty directories in zipfiles incorrectly.

On lines 1774 - 1778 in Lib/zipfile.py:

tgtdir = os.path.dirname(tgt)
if not os.path.exists(tgtdir):
    os.makedirs(tgtdir)
with open(tgt, 'wb') as fp:
    fp.write(zf.read(path))

In the case described above, tgt is 'dest/foo/' because the directory is empty. For non-empty directories, tgt would be a file in the directory i.e. 'dest/foo/a'. In the empty directory case, the directory will be created, but then opened as file (which will throw the error shown above).

When compressing the file with 'python -mzipfile -c', zipfile.py would not add empty directories to the zipfile. Hence, the zip file generated is empty.

This patch fixes both issues. In the decompression case, I utilize the Zipfile.extractall() function instead of extracting each file manually. The extractall() function handles empty directories correctly. For the compression case, I added a check to add an empty directory to the zip file.

[python-dev]

New changeset ffa5bfe75c3a by Serhiy Storchaka in branch '2.7':
Issue bpo-22201: Command-line interface of the zipfile module now correctly
http://hg.python.org/cpython/rev/ffa5bfe75c3a

New changeset 7b933005c492 by Serhiy Storchaka in branch '3.4':
Issue bpo-22201: Command-line interface of the zipfile module now correctly
http://hg.python.org/cpython/rev/7b933005c492

New changeset dc77ad3a17aa by Serhiy Storchaka in branch 'default':
Issue bpo-22201: Command-line interface of the zipfile module now correctly
http://hg.python.org/cpython/rev/dc77ad3a17aa

[serhiy-storchaka]

Applied first part of the patch. Thank you Ryan. For the rest please open separate issue.