Can't pickle partial functions

[danhs]

BPO	1398
Nosy	@jcea, @avassalotti, @florentx
Files	partial_bug.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2009-12-11.13:34:57.053>
created_at = <Date 2007-11-07.15:27:30.334>
labels = ['type-bug', 'library']
title = "Can't pickle partial functions"
updated_at = <Date 2011-12-11.01:28:15.741>
user = 'https://bugs.python.org/danhs'

bugs.python.org fields:

activity = <Date 2011-12-11.01:28:15.741>
actor = 'jcea'
assignee = 'none'
closed = True
closed_date = <Date 2009-12-11.13:34:57.053>
closer = 'alexandre.vassalotti'
components = ['Library (Lib)']
creation = <Date 2007-11-07.15:27:30.334>
creator = 'danhs'
dependencies = []
files = ['8705']
hgrepos = []
issue_num = 1398
keywords = []
message_count = 6.0
messages = ['57200', '57260', '71171', '71177', '95940', '96254']
nosy_count = 6.0
nosy_names = ['jcea', 'ngrilly', 'alexandre.vassalotti', 'pythonmeister', 'danhs', 'flox']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue1398'
versions = ['Python 2.5']

[danhs]

Creating a function using functools.partial results in a function which
cannot be pickled.

Attached is a small testcase.

[pythonmeister]

You are using an old protocol version

pickle.dumps(partial_f,2)

does the trick:

>>> pickle.dumps(partial_f,2)
'\x80\x02cfunctools\npartial\nq\x00)\x81q\x01}q\x02b.'

[ngrilly]

It seems using protocol version 2 is not enough:

>>> s = pickle.dumps(partial_f, 2)
>>> f = pickle.loads(s)
Traceback (most recent call last):
...
TypeError: type 'partial' takes at least one argument

Am I missing something?

[avassalotti]

I agree that this a bug. However, the liberal functools.partial
constructor makes it hard to pickle partial instances correctly.
Ideally, we would add the __getnewargs__ special method and be done with
it. But, this won't work in this case due to the *args and **kwargs
arguments of partial. Since pickle supports neither, we would need to
use apply(), which going to be removed in Python 3.0, as follow:

>>> import pickletools
>>>
pickletools.dis("c__builtin__\napply\n(cfunctools\npartial\n(c__main__\nf\nt(S'b'\nK\x01dtR.")
    0: c    GLOBAL     '__builtin__ apply'
   19: (    MARK
   20: c        GLOBAL     'functools partial'
   39: (        MARK
   40: c            GLOBAL     '__main__ f'
   52: t            TUPLE      (MARK at 39)
   53: (        MARK
   54: S            STRING     'b'
   59: K            BININT1    1
   61: d            DICT       (MARK at 53)
   62: t        TUPLE      (MARK at 19)
   63: R    REDUCE
   64: .    STOP

Unfortunately, pickle.Pickler cannot generate a such pickle stream. So
this bug is symptom of the bigger issue that classes with *args and/or
**kwargs argument cannot be made picklable.

[florentx]

It seems fixed on trunk.
It is fixed on Python 3.1 and 3.2, too.

With Python 2.6 I see the error:
>>> ./python partial_bug.py
TypeError: can't pickle partial objects

Documentation for "trunk" and "py3k" is OK?

[avassalotti]

Indeed. Jack Diederich added support for pickling partial functions in
r70931.