doctest doesn't allow duck-typing callables

[pitrou]

BPO	21740
Nosy	@gvanrossum, @tim-one, @rhettinger, @pitrou, @ezio-melotti, @bitdancer, @voidspace, @PCManticore, @1st1, @gvanrossum
Files	ducktest.py issue21740.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-12-08.20:01:17.325>
created_at = <Date 2014-06-12.20:59:12.925>
labels = ['type-feature', 'library']
title = "doctest doesn't allow duck-typing callables"
updated_at = <Date 2021-10-23.21:13:06.104>
user = 'https://github.com/pitrou'

bugs.python.org fields:

activity = <Date 2021-10-23.21:13:06.104>
actor = 'yaubi'
assignee = 'none'
closed = True
closed_date = <Date 2014-12-08.20:01:17.325>
closer = 'yselivanov'
components = ['Library (Lib)']
creation = <Date 2014-06-12.20:59:12.925>
creator = 'pitrou'
dependencies = []
files = ['35601', '35838']
hgrepos = []
issue_num = 21740
keywords = ['patch']
message_count = 11.0
messages = ['220384', '221039', '221040', '221073', '221145', '221917', '221926', '222167', '232324', '232325', '232688']
nosy_count = 11.0
nosy_names = ['gvanrossum', 'tim.peters', 'rhettinger', 'pitrou', 'ezio.melotti', 'r.david.murray', 'michael.foord', 'Claudiu.Popa', 'python-dev', 'yselivanov', 'Guido.van.Rossum']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue21740'
versions = ['Python 3.5']

[pitrou]

doctest uses inspect.isfunction() to detect callable objects on which to detect docstrings. Unfortunately, this prevents running doctests on functions which have been decorated to return other types of callables (for example numba's @jit decorator). In the attached example file, the wrapped "bar" function does not have its doctest executed.

It would be useful for doctest to be more flexible here, although I'm not sure what the exact criterion should be.

[ezio-melotti]

Would using callable() instead of inspect.isfunction() be ok?

[pitrou]

Ezio Melotti added the comment:

Would using callable() instead of inspect.isfunction() be ok?

I'm not sure, because it would also select classes. I guess we need
something a bit smarter.

[PCManticore]

How about using this?

diff -r 1e74350dd056 Lib/doctest.py
--- a/Lib/doctest.py    Tue Jun 17 22:27:46 2014 -0500
+++ b/Lib/doctest.py    Fri Jun 20 11:08:00 2014 +0300
@@ -984,7 +984,8 @@
             for valname, val in obj.__dict__.items():
                 valname = '%s.%s' % (name, valname)
                 # Recurse to functions & classes.
-                if ((inspect.isroutine(val) or inspect.isclass(val)) and
+
+                if ((inspect.isroutine(inspect.unwrap(val)) or inspect.isclass(val)) and
                     self._from_module(module, val)):
                     self._find(tests, val, valname, module, source_lines,
                                globs, seen)

This seems to work for the given example and if the decorator uses update_wrapper or @wraps.

[rhettinger]

I'm not sure, because it would also select classes.

Is there any reason to preclude classes? They could reasonably have docstrings that contain doctests.

[pitrou]

> I'm not sure, because it would also select classes.

Is there any reason to preclude classes? They could reasonably have docstrings that contain doctests.

I don't know. I just didn't want to change doctest behaviour too much,
but if people more knowledgeable than me about it feel it's ok, then all
the better.

[gvanrossum]

Class doctests are already supported separately, see https://docs.python.org/3/library/doctest.html#which-docstrings-are-examined

[PCManticore]

Here's a test patch which uses inspect.unwrap. Unfortunately, I can't test with numba, so I don't know if it works for that, but any decorated function which uses functools.update_wrapper or wraps should be detected by doctest.

[python-dev]

New changeset d22ca7496c54 by Yury Selivanov in branch 'default':
Issue bpo-21740: Support wrapped callables in pydoc. Patch by Claudiu Popa.
https://hg.python.org/cpython/rev/d22ca7496c54

[1st1]

Thank you for the patch, Claudiu!

[python-dev]

New changeset 12ef799a9a51 by Zachary Ware in branch 'default':
Issue bpo-21740: Fix module name in NEWS entry.
https://hg.python.org/cpython/rev/12ef799a9a51