Port importlib_metadata to Python 3.8

[warsaw]

BPO	34632
Nosy	@warsaw, @brettcannon, @rhettinger, @jaraco, @vstinner, @yan12125, @a-recknagel
PRs	bpo-34632: Add the importlib.metadata module #9327 bpo-34632: Add importlib.metadata #12547 bpo-34632: fix installation of importlib.metadata #13563 bpo-34632: fix test_importlib with installed CPython #13565 bpo-34632 fix buildbots and remove artifact #13566

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/warsaw'
closed_at = <Date 2019-09-12.11:00:18.434>
created_at = <Date 2018-09-11.18:16:43.409>
labels = ['3.8', 'library']
title = 'Port importlib_metadata to Python 3.8'
updated_at = <Date 2019-10-11.17:07:27.248>
user = 'https://github.com/warsaw'

bugs.python.org fields:

activity = <Date 2019-10-11.17:07:27.248>
actor = 'arne'
assignee = 'barry'
closed = True
closed_date = <Date 2019-09-12.11:00:18.434>
closer = 'jaraco'
components = ['Library (Lib)']
creation = <Date 2018-09-11.18:16:43.409>
creator = 'barry'
dependencies = []
files = []
hgrepos = []
issue_num = 34632
keywords = ['patch']
message_count = 20.0
messages = ['325043', '343440', '343441', '343445', '343457', '343459', '343460', '343461', '343462', '343465', '343480', '343481', '343483', '349415', '349416', '349421', '349423', '352105', '354466', '354469']
nosy_count = 7.0
nosy_names = ['barry', 'brett.cannon', 'rhettinger', 'jaraco', 'vstinner', 'yan12125', 'arne']
pr_nums = ['9327', '12547', '13563', '13565', '13566']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue34632'
versions = ['Python 3.8']

[warsaw]

https://importlib_metadata.rtfd.org

We're fleshing out the API and implementation in the standalone library, but once we're confident of the API and semantics, we will want to port this into Python 3.8.

[warsaw]

New changeset 1bbf7b6 by Barry Warsaw (Jason R. Coombs) in branch 'master':
bpo-34632: Add importlib.metadata (GH-12547)
1bbf7b6

[warsaw]

Thanks @jaraco! This is now merged into 3.8.

[vstinner]

Unhappy buildbot: AMD64 Fedora Rawhide Clang Installed 3.x

https://buildbot.python.org/all/#/builders/188/builds/302

Example:

0:00:28 load avg: 4.02 [182/422/1] test_importlib failed
Failed to import test module: test.test_importlib.test_main
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora.installed/build/target/lib/python3.8/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/home/buildbot/buildarea/3.x.cstratak-fedora.installed/build/target/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/home/buildbot/buildarea/3.x.cstratak-fedora.installed/build/target/lib/python3.8/test/test_importlib/test_main.py", line 6, in <module>
    import importlib.metadata
ModuleNotFoundError: No module named 'importlib.metadata'

[yan12125]

I got the same ModuleNotFoundError on Arch Linux and #13563 fixes it. I believe it can fix the issue on Fedora buildbots, too.

[jaraco]

I started trying to replicate the failure. I got as far as this Dockerfile:

FROM fedora:rawhide

RUN yum install -y clang make git

RUN git clone https://github.com/python/cpython
WORKDIR cpython
RUN ./configure
RUN make

And then running ./python [Tools/scripts/run_tests.py](https://github.com/python/cpython/blob/main/Tools/scripts/run_tests.py) test_importlib, but the tests fail due to zlib not being installed.

Sounds like yan12125 has the fix, so I'm shelving my investigation.

[jaraco]

New changeset c3738cf by Jason R. Coombs (Chih-Hsuan Yen) in branch 'master':
bpo-34632: fix installation of importlib.metadata (bpo-13563)
c3738cf

[yan12125]

Oops apparently my fix is incomplete. From the builder "AMD64 Fedora Rawhide Clang Installed 3.x" [1]:

ModuleNotFoundError: No module named 'test.test_importlib.data'

[1] https://buildbot.python.org/all/api/v2/logs/824407/raw

[yan12125]

By the way, I think Python.framework is not needed? 1bbf7b6#diff-206dc381e448d5121da9a6040a2b13c1

[yan12125]

I managed to create a setup similar to the buildbot builder "AMD64 Fedora Rawhide Clang Installed 3.x" [1] on Arch Linux. Running test_importlib on an installed CPython copy is fine now:

$ /usr/bin/python3.8 -m test.regrtest test_importlib
Run tests sequentially
0:00:00 load avg: 0.14 [1/1] test_importlib

== Tests result: SUCCESS ==

1 test OK.

Total duration: 1 sec 288 ms
Tests result: SUCCESS

I apologize for not checking things carefully and misunderstanding the issue on "AMD64 Fedora Rawhide Clang Installed 3.x".

[1] https://github.com/python/buildmaster-config/blob/master/master/custom/factories.py