pydocs fails for some C implemented classes

[serhiy-storchaka]

BPO	20204
Nosy	@loewis, @brettcannon, @amauryfa, @ncoghlan, @pitrou, @benjaminp, @ned-deily, @ezio-melotti, @asvetlov, @ericsnowcurrently, @serhiy-storchaka, @1st1
Files	tkinter_typespecs.patch: Workaround for the _tkinter module type_from_spec_module.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2015-03-19.07:25:59.424>
created_at = <Date 2014-01-09.12:47:35.821>
labels = ['interpreter-core', 'type-bug', 'expert-tkinter']
title = 'pydocs fails for some C implemented classes'
updated_at = <Date 2015-03-19.07:25:59.424>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2015-03-19.07:25:59.424>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2015-03-19.07:25:59.424>
closer = 'serhiy.storchaka'
components = ['Interpreter Core', 'Tkinter']
creation = <Date 2014-01-09.12:47:35.821>
creator = 'serhiy.storchaka'
dependencies = []
files = ['33375', '37958']
hgrepos = []
issue_num = 20204
keywords = ['patch', '3.4regression']
message_count = 12.0
messages = ['207733', '208989', '209003', '231674', '231695', '234269', '235165', '236871', '236879', '236919', '236936', '236941']
nosy_count = 15.0
nosy_names = ['loewis', 'brett.cannon', 'amaury.forgeotdarc', 'ncoghlan', 'pitrou', 'benjamin.peterson', 'ned.deily', 'ezio.melotti', 'Arfrever', 'asvetlov', 'python-dev', 'eric.snow', 'serhiy.storchaka', 'Robin.Schreiber', 'yselivanov']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue20204'
versions = ['Python 3.4', 'Python 3.5']

[serhiy-storchaka]

In 3.4 pydoc fails for the TkappType and TkttType names in the _tkinter module.

$ ./python -m pydoc _tkinter.TkappType
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/runpy.py", line 189, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/serhiy/py/cpython/Lib/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 2593, in <module>
    cli()
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 2558, in cli
    help.help(arg)
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 1840, in help
    elif request: doc(request, 'Help on %s:', output=self._output)
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 1578, in doc
    pager(render_doc(thing, title, forceload))
  File "/home/serhiy/py/cpython/Lib/pydoc.py", line 1555, in render_doc
    module = inspect.getmodule(object)
  File "/home/serhiy/py/cpython/Lib/inspect.py", line 610, in getmodule
    file = getabsfile(object, _filename)
  File "/home/serhiy/py/cpython/Lib/inspect.py", line 593, in getabsfile
    _filename = getsourcefile(object) or getfile(object)
  File "/home/serhiy/py/cpython/Lib/inspect.py", line 569, in getsourcefile
    filename = getfile(object)
  File "/home/serhiy/py/cpython/Lib/inspect.py", line 519, in getfile
    object = sys.modules.get(object.__module__)
AttributeError: __module__

And same for _tkinter.TkttType.

This issue can be easy fixed for the _tkinter module, but general solution is needed, because pydoc in 3.4 still can be broken for third-party code.

[1st1]

See issue bpo-20372 -- fix for the 'inspect.getfile' function.

[serhiy-storchaka]

With issue bpo-20372 patch pydoc no longer raise an exception, but it also doesn't produce useful output. In 3.3 it prints more details.

[serhiy-storchaka]

The problem not in pydoc or inspect itself. In Python 3.3 _tkinter.TkappType has the __module__ attribute:

>>> import _tkinter
>>> _tkinter.TkappType.__module__
'builtins'

Something was changed in 3.4 and builtin classes without dot in qualified name no longer have the __module__ attribute.

[serhiy-storchaka]

_tkinter.TkappType.__flags__ is different.

In 3.3: 0b1000001000000000000
In 3.4: 0b1000001001000000000

The difference is Py_TPFLAGS_HEAPTYPE. This is the consequence of the change made in bpo-15721.

[serhiy-storchaka]

I think we should add a check to ensure than heap types have the __module__ attribute.

[serhiy-storchaka]

Here is a patch which adds a warning to PyType_FromSpec and PyType_FromSpecWithBases if type spec name doesn't contain module name. In conjunction with tkinter_typespecs.patch it should fix the issue.

[serhiy-storchaka]

What type of warning is more preferable here? It will be emitted at import. E.g.:

$ ./python -Wall
Python 3.5.0a1+ (default:28ba862036cc+, Feb 28 2015, 11:01:23) 
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tkinter
_frozen_importlib:321: SyntaxWarning: builtin type tkapp has no the __module__ attribute
_frozen_importlib:321: SyntaxWarning: builtin type tktimertoken has no the __module__ attribute

DeprecationWarning? RuntimeWarning?

[ncoghlan]

The case for RuntimeWarning: the object isn't necessarily *broken* as such (most things will still work), but pickling and some introspection features may not work properly.

The case for DeprecationWarning: breaking picking and introspection is bad when the fix (setting __module__) is straightforward, so this should eventually become an AttributeError:

AttributeError: __module__ not set on builtin type tkapp

My own preference is for the latter - eventually making it a hard requirement to specify the module name properly. Even true builtins officially live in the builtins module:

    >>> str.__module__
    'builtins'

[serhiy-storchaka]

It also can be ImportWarning (warnings triggered during the process of importing a module (ignored by default)).

[ncoghlan]

ImportWarning is slightly different - it's aimed at issues that happen during the operation of the import machinery itself.

This isn't that - it's a warning related to the extension module actually initialising itself, so it's akin to a warning issued due to the behaviour of top level module source code, rather than by the import system.