modulefinder uses wrong CodeType signature in .replace_paths_in_code()

[malemburg]

BPO	21707
Nosy	@malemburg, @bitdancer, @berkerpeksag
Files	issue21707.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/berkerpeksag'
closed_at = <Date 2014-07-07.12:00:51.725>
created_at = <Date 2014-06-10.16:59:48.897>
labels = ['type-bug', 'library']
title = 'modulefinder uses wrong CodeType signature in .replace_paths_in_code()'
updated_at = <Date 2014-07-07.18:30:44.632>
user = 'https://github.com/malemburg'

bugs.python.org fields:

activity = <Date 2014-07-07.18:30:44.632>
actor = 'python-dev'
assignee = 'berker.peksag'
closed = True
closed_date = <Date 2014-07-07.12:00:51.725>
closer = 'berker.peksag'
components = ['Library (Lib)']
creation = <Date 2014-06-10.16:59:48.897>
creator = 'lemburg'
dependencies = []
files = ['35587']
hgrepos = []
issue_num = 21707
keywords = ['patch', '3.4regression']
message_count = 5.0
messages = ['220174', '220176', '220334', '222451', '222495']
nosy_count = 4.0
nosy_names = ['lemburg', 'r.david.murray', 'python-dev', 'berker.peksag']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue21707'
versions = ['Python 3.4', 'Python 3.5']

[malemburg]

Here's the code:

    def replace_paths_in_code(self, co):
        ...
        return types.CodeType(co.co_argcount, co.co_nlocals, co.co_stacksize,
                         co.co_flags, co.co_code, tuple(consts), co.co_names,
                         co.co_varnames, new_filename, co.co_name,
                         co.co_firstlineno, co.co_lnotab,
                         co.co_freevars, co.co_cellvars)

Compare this to the code_new() C API doc string (and code):

code(argcount, kwonlyargcount, nlocals, stacksize, flags, codestring,\n\
      constants, names, varnames, filename, name, firstlineno,\n\
      lnotab[, freevars[, cellvars]])

The kwonlyargcount is missing in the call used in .replace_paths_in_code().

The bug surfaces when trying to use the "-r" option of the freeze.py tool:

Traceback (most recent call last):
  File "freeze.py", line 504, in <module>
    main()
  File "freeze.py", line 357, in main
    mf.import_hook(mod)
  File "/home/lemburg/egenix/projects/PyRun/tmp-3.4-ucs2/lib/python3.4/modulefinder.py", line 123, in import_hook
    q, tail = self.find_head_package(parent, name)
  File "/home/lemburg/egenix/projects/PyRun/tmp-3.4-ucs2/lib/python3.4/modulefinder.py", line 179, in find_head_package
    q = self.import_module(head, qname, parent)
  File "/home/lemburg/egenix/projects/PyRun/tmp-3.4-ucs2/lib/python3.4/modulefinder.py", line 272, in import_module
    m = self.load_module(fqname, fp, pathname, stuff)
  File "/home/lemburg/egenix/projects/PyRun/tmp-3.4-ucs2/lib/python3.4/modulefinder.py", line 303, in load_module
    co = self.replace_paths_in_code(co)
  File "/home/lemburg/egenix/projects/PyRun/tmp-3.4-ucs2/lib/python3.4/modulefinder.py", line 569, in replace_paths_in_code
    consts[i] = self.replace_paths_in_code(consts[i])
  File "/home/lemburg/egenix/projects/PyRun/tmp-3.4-ucs2/lib/python3.4/modulefinder.py", line 575, in replace_paths_in_code
    co.co_freevars, co.co_cellvars)
TypeError: an integer is required (got type bytes)

[malemburg]

The fix is easy. Simply change the call to:

        return types.CodeType(co.co_argcount, co.co_kwonlyargcount,
                              co.co_nlocals, co.co_stacksize,
                              co.co_flags, co.co_code, tuple(consts),
                              co.co_names, co.co_varnames,
                              new_filename, co.co_name,
                              co.co_firstlineno, co.co_lnotab,
                              co.co_freevars, co.co_cellvars)

(ie. add the missing argument)

[berkerpeksag]

Here's a patch with a test.

[python-dev]

New changeset 59921d2f023c by Berker Peksag in branch '3.4':
Issue bpo-21707: Add missing kwonlyargcount argument to ModuleFinder.replace_paths_in_code().
http://hg.python.org/cpython/rev/59921d2f023c

New changeset 4b6798e74dcf by Berker Peksag in branch 'default':
Issue bpo-21707: Merge with 3.4.
http://hg.python.org/cpython/rev/4b6798e74dcf

[python-dev]

New changeset f8deaac44ed4 by Berker Peksag in branch '3.4':
Issue bpo-21707: Fix tests on Windows.
http://hg.python.org/cpython/rev/f8deaac44ed4

New changeset e66c387da81b by Berker Peksag in branch 'default':
Issue bpo-21707: Merge with 3.4.
http://hg.python.org/cpython/rev/e66c387da81b