Loading Meteor metric in HF evaluate module crashes due to datasets import issue

[dsashulya]

Describe the bug

When using python3.9 and evaluate module loading Meteor metric crashes at a non-existent import from datasets.config in datasets v2.14

Steps to reproduce the bug

from evaluate import load
meteor = load("meteor")

produces the following error:

  from datasets.config import importlib_metadata, version
ImportError: cannot import name 'importlib_metadata' from 'datasets.config' (<path_to_project>/venv/lib/python3.9/site-packages/datasets/config.py)

Expected behavior

datasets of v2.10 has the following workaround in config.py:

if PY_VERSION < version.parse("3.8"):
    import importlib_metadata
else:
    import importlib.metadata as importlib_metadata

However, it's absent in v2.14 which might be the cause of the issue.

Environment info

• datasets version: 2.14.4
• Platform: macOS-13.5-arm64-arm-64bit
• Python version: 3.9.6
• Huggingface_hub version: 0.16.4
• PyArrow version: 12.0.1
• Pandas version: 2.0.3
• Evaluate version: 0.4.0

[mariosasko]

Our minimal Python version requirement is 3.8, so we dropped importlib_metadata.

Feel free to open a PR in the evaluate repo to replace the problematic import with

if PY_VERSION < version.parse("3.8"):
    import importlib_metadata
else:
    import importlib.metadata as importlib_metadata

[SajjadPSavoji]

Any idea when you guys will release the next version which deals with this problem?
I'm still having the same issue with py 3.10 when I install the lib with pip.
I'm assuming that it has not yet been updated since the merge was 3 days ago.

[mariosasko]

Yes, this requires a new evaluate release (cc @lvwerra for this).

In the meantime, you can get the fixed version by installing evaluate from main: pip install git+https://github.com/huggingface/evaluate.git

[lvwerra]

I'll aim for a release this week!