tarfile.open(mode = 'r:*', ignore_zeros = True) has 50% chance failed to open compressed tars?

[SilverFox]

BPO	28449
Nosy	@gustaebel, @serhiy-storchaka, @RoliSoft
PRs	[Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
Files	tarfile_ignore_zeros_auto.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2016-10-30.18:59:50.846>
created_at = <Date 2016-10-15.09:07:44.373>
labels = ['3.7', 'type-bug', 'library']
title = "tarfile.open(mode = 'r:*', ignore_zeros = True) has 50% chance failed to open compressed tars?"
updated_at = <Date 2017-03-31.16:36:31.712>
user = 'https://bugs.python.org/SilverFox'

bugs.python.org fields:

activity = <Date 2017-03-31.16:36:31.712>
actor = 'dstufft'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2016-10-30.18:59:50.846>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2016-10-15.09:07:44.373>
creator = 'Silver Fox'
dependencies = []
files = ['45119']
hgrepos = []
issue_num = 28449
keywords = ['patch']
message_count = 4.0
messages = ['278711', '278767', '278773', '279740']
nosy_count = 5.0
nosy_names = ['lars.gustaebel', 'python-dev', 'serhiy.storchaka', 'Silver Fox', 'RoliSoft']
pr_nums = ['552']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue28449'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.6', 'Python 3.7']

[SilverFox]

Seems all tarfile.open will try all compress method(including raw) in random order, and ignore_zeros also hide the error during detecting compress method. So raw is used if tested before the rigth compress method, which is wrong.

But there is no warning that the 2 args can not be used together in docs.

[RoliSoft]

For anyone finding this bug through Google before it is fixed, a workaround could be to monkeypatch the OPEN_METH dict with an OrderedDict:

	tarfile.TarFile.OPEN_METH = OrderedDict()
	tarfile.TarFile.OPEN_METH['gz']  = 'gzopen'
	tarfile.TarFile.OPEN_METH['bz2'] = 'bz2open'
	tarfile.TarFile.OPEN_METH['xz']  = 'xzopen'
	tarfile.TarFile.OPEN_METH['tar'] = 'taropen'

[serhiy-storchaka]

Proposed patch fixes the issue.

Existing test was passed by accident -- compressed tarfiles were too short for false detecting.

[python-dev]

New changeset f108e063e299 by Serhiy Storchaka in branch '3.5':
Issue bpo-28449: tarfile.open() with mode "r" or "r:" now tries to open a tar
https://hg.python.org/cpython/rev/f108e063e299

New changeset e2dd0f48e643 by Serhiy Storchaka in branch '2.7':
Issue bpo-28449: tarfile.open() with mode "r" or "r:" now tries to open a tar
https://hg.python.org/cpython/rev/e2dd0f48e643

New changeset de8e83262644 by Serhiy Storchaka in branch '3.6':
Issue bpo-28449: tarfile.open() with mode "r" or "r:" now tries to open a tar
https://hg.python.org/cpython/rev/de8e83262644

New changeset 220c70519958 by Serhiy Storchaka in branch 'default':
Issue bpo-28449: tarfile.open() with mode "r" or "r:" now tries to open a tar
https://hg.python.org/cpython/rev/220c70519958