setup.py shouldn't import the package it installs

[greut]

There is a 🐔 and 🥚 problem regarding how setup.py works. the version number should be parsed from the text file rather than imported via python mechanism.

    File "setup.py", line 26, in <module>
      from markdown import __version__, __version_info__
  ModuleNotFoundError: No module named 'markdown'

[greut]

cf #791

[mitya57]

The same thing now happens in Travis BTW, see https://travis-ci.org/Python-Markdown/markdown/builds/531020063.

I am not yet sure what causes it, though — setup.py imports Markdown but Markdown does not import setup.py so there is no cyclic dependency.

[facelessuser]

I guess there is one way to be sure, we can perform an experiment. We can move the version stuff to a separate file, and then manually import just that file directly.

[mitya57]

I will try to do that a bit later.

Also I don't know if it will be useful or not, but modern setuptools supports specifying the version in setup.cfg in declarative format:

version = attr: markdown.__version__

or

version = file: VERSION.txt

The former worked for me some time ago, need to check again. Upd: still works in https://travis-ci.org/retext-project/pymarkups/builds/502474546 although I do not use Tox.

[facelessuser]

If you can declare it like you mentioned, and have it exist in one place, maybe that is preferable.

[mitya57]

I was able to fix Travis tests by setting PYTHONPATH to the current directory: https://github.com/mitya57/Python-Markdown/commit/291a71239a245371a0ad455ec6a279b02814c235. Strangely, I was not able to reproduce the original problem locally, even when using tox.

@greut How exactly are you calling setup.py? Can you check if setting PYTHONPATH=$(pwd) fixes the issue for you too?

[greut]

@mitya57 I'm installing it via Requirements-Builder and the git+https URL. Pip fails into the _vendor/pep517/_in_process.py@get_requires_for_build_wheel. Something like, pypa/pip#6163

Is the project fully PEP517 compliant?

edit strangely django/django installs just fine and it seems to perform same kind of 🐔 and 🥚. So it should be possible.

[greut]

btw python3 -m pep517.check fails

[waylan]

This is definitely a PEP517 issue, which makes this a duplicate of #791. However, I'm closing that as this better identifies the issue.

Under Build Environment, PEP517 states:

A build backend MUST be prepared to function in any environment which meets the above criteria. In particular, it MUST NOT assume that it has access to any packages except those that are present in the stdlib, or that are explicitly declared as build-requirements.

Note that the cwd is not listed among the locations which we can assume to have access to (this is the change which is breaking things. We have always been able to make that assumption in the past). In a later subsection, In-Tree Build Backends, there is a provision defined for giving the build environment access to the source tree:

Projects can specify that their backend code is hosted in-tree by including the backend-path key in pyproject.toml. This key contains a list of directories, which the frontend will add to the start of sys.path when loading the backend, and running the backend hooks.

Of course, the source code for the project is not technically part of the build environment backend, in Python-Markdown's case, but we could use it that way if we want to leave the version identification code as-is.

@mitya57 demonstrated how we could define the version in setup.cfg in a comment above. Or we could define the version in setup.py itself. However those are using the old system which Python seems to be leaving behind.

I think we should probably explore adopting the new system and use pyproject.toml. However, that means defining the version in the pyproject.toml file. If we want to have the version accessible from markdown.__version__, then we would likely need to import disutils from the markdown lib and retrieve the disutls registered version of markdown, which we could assign to markdown.__version__. Unless there is some other way to do that, that I'm not aware of.

Regardless of what solution we use, we want to only ever define the version in one location. Statically defining the version in multiple locations is only asking for them to get out of sync. I'm having a hard time believing the new system doesn't have a simple API for this already, but I haven't found one.

[waylan]

Interestingly, when I had first learned of pyproject.toml, I had considered moving markdown to using it. That would have made sense for the 3.0 release (or even the 3.1 release). But I decided against it for the reason that I dislike the way versions are defined. Our current setup requires that we define the individual pieces of the __version_info__ tuple and those pieces are programmatically assembled into a valid version number. Personally, I never remember the specifics of how those pieces come together (especially for alpha, beta, rc and the like). Any change would loose that. But if people can't install the lib, then it doesn't matter what format the version number is in, valid or not.

[mitya57]

In a later subsection, In-Tree Build Backends, there is a provision defined for giving the build environment access to the source tree

I think in-tree backends were not meant to be used this way. The PEP mentions only two use cases: backends themselves and wrappers around existing backends where the wrapper is too project-specific. I think our case does not fall into any of these two.

I think we should probably explore adopting the new system and use pyproject.toml. However, that means defining the version in the pyproject.toml file.

Can you give a pointer to some documentation about this, or a project that specifies a version in pyproject.toml? I have not heard about this yet.

There is flit build system that allows one to specify metadata in pyproject.toml but it does not allow to specify version there…

If we want to have the version accessible from markdown.__version__, then we would likely need to import disutils from the markdown lib and retrieve the disutls registered version of markdown, which we could assign to markdown.__version__.

By disutils do you mean distutils? If yes, then I think it does not support getting the version of installed package. One can use pkg_resources to get the version, but pkg_resources has a big performance impact (see #790).