Restore loading of TYPE_INT64 in marshal

[serhiy-storchaka]

BPO	32011
Nosy	@serhiy-storchaka
PRs	bpo-32011: Revert "Issue #15480: Remove the deprecated and unused TYP… #4381 [3.6] bpo-32011: Revert "Issue GH-15480: Remove the deprecated and unused TYPE_INT64 code from marshal." (GH-4381) #4405

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2017-11-15.16:06:19.782>
created_at = <Date 2017-11-12.17:50:16.906>
labels = ['extension-modules', 'type-bug', '3.7']
title = 'Restore loading of TYPE_INT64 in marshal'
updated_at = <Date 2017-11-15.16:06:19.781>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2017-11-15.16:06:19.781>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2017-11-15.16:06:19.782>
closer = 'serhiy.storchaka'
components = ['Extension Modules']
creation = <Date 2017-11-12.17:50:16.906>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 32011
keywords = ['patch']
message_count = 4.0
messages = ['306128', '306130', '306276', '306281']
nosy_count = 1.0
nosy_names = ['serhiy.storchaka']
pr_nums = ['4381', '4405']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue32011'
versions = ['Python 3.6', 'Python 3.7']

[serhiy-storchaka]

TYPE_INT64 is supported in the marshal module since Python 1.5. Since its use causes instability of marhalled files produced on different platforms, the marshal module in Python 3.3 no longer use it when write new files (see bpo-15466). And in Python 3.4 it stopped reading files containing the TYPE_INT64 code (see bpo-15480).

This is backward incompatible change. Python 2.7 still produce files containing the TYPE_INT64 code (though this code is rare, because integers in ranges -9223372036854775808..-2147483649 and 2147483648..9223372036854775807 are rare), and there may be marshal files produced by older versions of Python 3, Python 2 and Python 1. Formally Python 3 supports marshal formats produced by all previous versions, except the code objects and the TYPE_INT64 code.

Supporting loading the TYPE_INT64 code don't create problems like using it for saving data. But it is needed for backward compatibility.

I think that removing the TYPE_INT64 code was a mistake.

[serhiy-storchaka]

Example:

$ python2 -c 'import sys, marshal; marshal.dump(1234567890, sys.stdout)' | python3 -c 'import sys, marshal; print(marshal.load(sys.stdin.buffer))'
1234567890

$ python2 -c 'import sys, marshal; marshal.dump(123456789012345, sys.stdout)' | python3 -c 'import sys, marshal; print(marshal.load(sys.stdin.buffer))'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ValueError: bad marshal data (unknown type code)

python2 -c 'import sys, marshal; marshal.dump(12345678901234567890, sys.stdout)' | python3 -c 'import sys, marshal; print(marshal.load(sys.stdin.buffer))'
12345678901234567890

PR 4381 reverts this change and simplifies the restored code.

[serhiy-storchaka]

New changeset 00987f6 by Serhiy Storchaka in branch 'master':
bpo-32011: Revert "Issue bpo-15480: Remove the deprecated and unused TYPE_INT64 code from marshal." (bpo-4381)
00987f6

[serhiy-storchaka]

New changeset d15bb5f by Serhiy Storchaka (Miss Islington (bot)) in branch '3.6':
bpo-32011: Revert "Issue #59685: Remove the deprecated and unused TYPE_INT64 code from marshal." (GH-4381) (bpo-4405)
d15bb5f