PyPI packages failing build or tests

[jayvdb]

This is a dumping ground for build or test failures encountered. Not nearly as systematic as Nuitka/Nuitka#522 because I cant use OBS due to #164 , but I hope to resolve that soon.

All tests are run with pytest as an external import unless otherwise noted. I am focusing on core compiled libraries, as they are most likely to fail silently, and a few other CLI deps of mine.

Upstream rejected solutions:

• cmd2 Allow readline.__file__ to be missing python-cmd2/cmd2#802 / Gracefully degrade when loading libreadline fails python-cmd2/cmd2#807 They even deleted three of my comments explaining the problem to them, and then deleting another two comments, one which was merely noting that they deleted my comments.

Build & test tips

• Click pytest.main(['-v', '/path/to/Click-7.0/tests/', '-k', 'not test_cli_as_module'])
• pygit2, but incompatible with pyrsistent Static symbol moduleinit clashes with pyrsistent and evdev libgit2/pygit2#951
• PyYAML needs Cython pre-installed in order to build the DSO, and running test suite requires being in the project root dir, and running pytest.main(['-v', './tests/lib3']) passes all tests.

Upstream failures

• ruamel.yaml.clib builds ok, but getting the core tests passing is a bit finicky, and some fail only because ruamel.yaml.main uses __file__ , raised as https://bitbucket.org/ruamel/yaml/issues/324/dependency-on-__file__ . Most usage is unlikely to be effected as this is the CLI of ruamel.yaml
• toolz/cytoolz __file__ Ignore missing __file__ pytoolz/toolz#476
• decorator Use of __file__ breaks PyOxidizer micheles/decorator#73 (merged)
• httpbin Use importlib.resources instead of __file__ postmanlabs/httpbin#584
• Django, https://code.djangoproject.com/ticket/30950 - note installation of Django breaks pytest if pytest-webtest or pytest-django (or other similar plugins) is in the sys.path
• ply Avoid __file__ dabeaz/ply#216 effecting pycparser Avoid __file__ eliben/pycparser#355 , also effecting cffi in the built binary (but not in the packaging rule stages) cffi also fails because it needs Python.h
• cytoolz tlz._build_tlz uses __file__ Ignore missing __file__ pytoolz/toolz#476
• yaspin - fixed on master
• orderedmultidict - Regression with PyInstaller (1.0.1 compared to 1.0) gruns/orderedmultidict#25, pin to <1.0.1 to avoid breakage
• kombu and cell (and cl) Avoid use of __file__ celery/kombu#1121 / Avoid use of __file__ auvipy/cell#21
• boltons Avoid use of __file__ mahmoud/boltons#229
• hypothesis Avoid use of __file__ HypothesisWorks/hypothesis#2196 (released)
• docker compose Avoid use of __file__ docker/compose#7011
• pycountry (all below are __file__ to be raised upstream)
• python-whois
• httplib2 (similar to certifi problem)
• Babel Replace __file__ with newer APIs python-babel/babel#674
• pytz
• jsonschema
• tinycss2
• pyx
• pyglet
• netaddr Avoid use of __file__ netaddr/netaddr#188

Known failures

• psutil tests mostly fail because of the use sys.executable -c . Using import hooks to add __file__ helps all tests pass. pytest.main(['-v', '/path/to/psutil/tests', '-k', 'not (test_process or TestProcessUtils or TestScripts or TestTerminatedProcessLeaks)']) removes most of those.
  [ ] coverage has lots of __file__ problems, and likely too entrenched to be fixed any time soon. And test failures due to use of sys.executable -c '..'

Unknown failures

• peewee needs to be tested using runtests.py in the project root, and it fails only tests.fields.TestTimestampField, in what seems to be a timezone offset problem as the day field is off by one. Need to retest locally with CPython directly.
• requests, after hacking around certifi, and even getting cryptography extensions working(c.f. cryptography modules not built due to PEP 517 #170), test_https_warnings still fails and 2 of 4 of the "test_errors" fail pytest.main(['-v', '/path/to/requests-2.22.0/tests/', '-k', 'not TestTimeout and not test_errors and not test_proxy_error']) is my current recipe. fixture httpbin_ca_bundle = None is probably the problem - something is probably wrong with my httpbin.
• gevent tests fail to get collected due to use of sys.executable -c; need to get its greentest running under PyOxidizer to evaluate properly
• lz4 had one failure test_roundtrip_1[data1-4-True-True-True-0-store_size0] , and then finished at 37%
• blist, errors NameError: name 'cmp' is not defined when run under pytest. Need to retest with unittest, as it is passing at https://build.opensuse.org/package/show/devel:languages:python/python-blist
• aiodns one failure test_query_a_bad
• aiohttp[speedups] tests use __file__, but there are also a four other failures, mostly related to network responses. test_fingerprint_mismatch, test_aiohttp_request_ctx_manager_not_found, test_recv_protocol_error[pyloop], test_constructor[pyloop], TestCookieJarSafe.test_expires, TestCookieJarSafe.test_max_age
• Flask - only three tests fail: test_scriptinfo fails but looks due to test runner path fiddling, test_main_module_paths and test_aborting. (This looks good to enough to say that PyOxidizer is Flask compatible.)
• pycares - 9 quite significant failures

OK

• PySocks
• idna
• persistent
• multidict
• msgpack
• cmarkgfm
• yarl
• cChardet

[jstasiak]

netaddr has this fixed I think, let me know if it still causes trouble.

[AvdN]

ruamel.yaml 0.17.0 no longer throws an exception based on __file__. If you don't rely on any plug-ins you should be able to oxidize your program as is.

https://sourceforge.net/p/ruamel-yaml/tickets/324/

[sevein]

jsonschema is now using importlib.resources: https://github.com/Julian/jsonschema/pull/873/files (v4.2.0).