markdown does not import on ArchLinux

[mitya57]

I am forwarding here the bug I have received in retext-project/retext#458:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    import markdown
  File "/usr/lib/python3.7/site-packages/markdown/__init__.py", line 67, in <module>
    __version__ = _get_version()
  File "/usr/lib/python3.7/site-packages/markdown/__init__.py", line 64, in _get_version
    return str(packaging.version.Version(v))
AttributeError: module 'pkg_resources._vendor.packaging' has no attribute 'version'

I looked at ArchLinux PKGBUILD file and found out that they are patching setuptools to use system versions of modules such as packaging, instead of vendored versions:

https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/python-setuptools#n26

Now I realized that pkg_resources.extern.packaging is a non-documented module, and we cannot be sure that it exists on all platforms or that it will not be removed in the future. So probably we should switch to the official version of packaging or at least do something like this:

try:
    import packaging
except ImportError:
    from pkg_resources.extern import packaging

[eli-schwartz]

Actually, markdown imports just fine on archlinux... We do not package markdown 3.1 at all, my assumption is because this bug makes it impossible to build any new version of it.

Ping @keenerd

[mitya57]

Ah, so the reporter of retext-project/retext#458 was using setuptools from ArchLinux with markdown 3.1 from PyPI.

So this (upstream) bug is valid, but the bug I asked them to file against distro package is invalid. Sorry for the confusion.

[mitya57]

Another option may be using pkg_resources.parse_version() which is public and documented API:

https://setuptools.readthedocs.io/en/latest/pkg_resources.html#parsing-utilities

https://github.com/pypa/setuptools/blob/3adda966e2e76ad59cf79fd8bdeab77fb11d308f/pkg_resources/__init__.py#L127-L131

[eli-schwartz]

That seems reasonable. pkg_resources is kind of slow to import just for that, but markdown.core will import it either way (and is itself imported in __init__) so this won't add anything new.

[waylan]

Another option may be using pkg_resources.parse_version() which is public and documented API

My recollection is that that won't work as packaging normalizes the input to a valid version while pkg_resources does not. But I may be remembering that incorrectly. Or it could be that the differences only existed in older python versions. The documentation is not clear for either on how that works. I think I passed both some random (likely) invalid versions and compared the output, but again, my memory is fuzzy on that.

If it works, then by all means.

[waylan]

I just checked the source code and using pkg_resources.parse_version() won't help. All it does is import and use packaging. So ArchLinux has broken a documented API in pkg_resources. This is an upstream problem. I'm not inclined to do anything. ArchLinux can fix this themselves.

As an aside, we are not going to call pkg_resources.parse_version() as it uses LegacyVersion as a fallback. We do not want that behavior. That is the different behavior I couldn't remember in my previous comment.

[eli-schwartz]

The docs say it should return a PEP 440 version, PEP 440 says versions should be normalized, and as of today pkg_resources.parse_version() is a convenience wrapper for, essentially:

import packaging
return packaging.version.Version(...)

Arch Linux most definitively does not break parse_version... that function correctly imports packaging, but does not do it via pkg_resources.extern.

[mitya57]

I just checked the source code and using pkg_resources.parse_version() won't help. All it does is import and use packaging. So ArchLinux has broken a documented API in pkg_resources.

You are wrong — ArchLinux did not break documented API. They replace the import on top of the file, so parse_version works, but uses external version of packaging module rather than the bundled version.

[mitya57]

As an aside, we are not going to call pkg_resources.parse_version() as it uses LegacyVersion as a fallback. We do not want that behavior.

That is understandable. Are you fine with one of the original proposals then — either adding a dependency on packaging, or using the fallback?

[eli-schwartz]

@mitya57 Also note the comments above that, explaining exactly why we implemented the precise solution which we did. :)

setuptools is broken and breaks pip when devendored, because its vendoring solution is inferior to the one pip uses. In fact, the Version object which pkg_resources returns cannot be compared to objects created by packaging.version.Version (unless you are on Arch Linux, I guess)