ImportError: cannot import name yaml2

[ojos]

The following errors occured in Python2 environment.

> invoke
Traceback (most recent call last):
  File "/Users/ojos/Workspace/hoge/python/bin/invoke", line 7, in <module>
    from invoke.main import program
  File "/Users/ojos/Workspace/hoge/python/lib/python2.7/site-packages/invoke/__init__.py", line 2, in <module>
    from .collection import Collection  # noqa
  File "/Users/ojos/Workspace/hoge/python/lib/python2.7/site-packages/invoke/collection.py", line 7, in <module>
    from .config import merge_dicts
  File "/Users/ojos/Workspace/hoge/python/lib/python2.7/site-packages/invoke/config.py", line 11, in <module>
    from .vendor import yaml2 as yaml

There is not invoke/vender/yaml2 package to version 0.12.1.
Please include.

[raimonesteve]

I found same error. It's when install with "pip install invoke" and 0.12.1. Before version, 0.12.0 install successfully.

[raimonesteve]

I try to install with "python setup.py install" and yaml2 directory found it, but not yaml3.

~/virtualenv/invprova$ ls lib/python2.7/site-packages/invoke-0.12.1-py2.7.egg/invoke/vendor/ fluidity/ __init__.py __init__.pyc lexicon/ six.py six.pyc yaml2/

I'm working with py2.7.

[ojos]

I installed in pip 7.1.2(python 2.7)

[denisgarci]

Same here with Python 2. I don't have the issue on 0.12.0.

[noirbizarre]

Same here, downgraded on 0.12.0 where the problem does not occurs.

[Benoss]

Same here
python 2.7.6
pip 8.0.2
fresh virtualenv

>>> import invoke
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/benoit/.virtualenvs/test_invoke2/local/lib/python2.7/site-packages/invoke/__init__.py", line 2, in <module>
    from .collection import Collection  # noqa
  File "/home/benoit/.virtualenvs/test_invoke2/local/lib/python2.7/site-packages/invoke/collection.py", line 7, in <module>
    from .config import merge_dicts
  File "/home/benoit/.virtualenvs/test_invoke2/local/lib/python2.7/site-packages/invoke/config.py", line 11, in <module>
    from .vendor import yaml2 as yaml
ImportError: cannot import name yaml2

[alfakini]

Same problem here while using streamparse, I downgraded on 0.12.0 and everything is now working.

$ sparse run -n test_topology
Traceback (most recent call last):
  File "/Users/user/miniconda2/bin/sparse", line 11, in <module>
    sys.exit(main())
  File "/Users/user/miniconda2/lib/python2.7/site-packages/streamparse/cli/sparse.py", line 51, in main
    load_subparsers(subparsers)
  File "/Users/user/miniconda2/lib/python2.7/site-packages/streamparse/cli/sparse.py", line 27, in load_subparsers
    module = importlib.import_module('streamparse.cli.{}'.format(mod_name))
  File "/Users/user/miniconda2/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/Users/user/miniconda2/lib/python2.7/site-packages/streamparse/cli/jar.py", line 9, in <module>
    from invoke import run
  File "/Users/user/miniconda2/lib/python2.7/site-packages/invoke/__init__.py", line 2, in <module>
    from .collection import Collection  # noqa
  File "/Users/user/miniconda2/lib/python2.7/site-packages/invoke/collection.py", line 7, in <module>
    from .config import merge_dicts
  File "/Users/user/miniconda2/lib/python2.7/site-packages/invoke/config.py", line 11, in <module>
    from .vendor import yaml2 as yaml
ImportError: cannot import name yaml2

[aaron-trout]

+1 :D

[RazerM]

The problem is that wheels don't execute setup.py, which excludes either yaml2 or yaml3 at runtime. So we can surmise that the wheel was built on Python 3.

You can get around this by disabling the wheel when installing invoke with pip

pip install --no-binary invoke invoke

[RazerM]

I looked at the wheel for 0.12.0 and it has both yaml2 and yaml3 included. I don't know how it was generated unless it was done before the code to exclude yaml2 or yaml3 was added.

[jenisys]

REGARDING: wheel
The wheel may be currently the problem, but the fault location is actually in the "setup.py" file which is lacking the necessary description when you build/publish wheels. You need something like the following in "setup.py" to build your "universal" wheels correctly:

# -- FILE: setup.py
...
setup(
    ...
    extras_require={
        # -- SUPPORT-WHEELS: Extra packages for Python2.6
        # SEE: https://bitbucket.org/pypa/wheel/ , CHANGES.txt (v0.24.0)
        ':python_version=="2.7"': py27_extra,
        ':python_version=="3.3"': py33_extra,
    },
    ...
)

In case of invoke, the following line is the problem:

# -- FILE: setup.py
# PACKAGING-/PUBLISHER-VIEW, but not necessarily PACKAGE-INSTALLER VIEW:
exclude = ['*.yaml3' if sys.version_info[0] == 2 else '*.yaml2']

If you build universal wheels for both python2 and python3, the wheel must include everything. Even stuff that is not needed for python3, for example. But it would be nice, if the wheel would only the install parts (meaning either yaml2 or yaml3 variant).

[bitprophet]

Well that's great :( Yes, this is due to a "fix" added with that exclude line recently, in...#254 / #255 (though there had been reports of the earlier issue prior). Thanks to all for reporting & digging!

The problem of universal wheels makes sense offhand. My memory is rusty but I don't think we actually need universal wheels, they are just a convenience for projects where the 2 + 3 versions are identical, which is no longer the case for us (or rather, it causes the problems reported in #254).

I'm updating my packaging tasks to do some basic testing of build & install at CI time, so I'll try to replicate it there & test out some fixes, either non-universal wheels (which I think means I'll need to run the build task under both Python versions? will see) or the extras_require that @jenisys dug up (thanks!).

[bitprophet]

Also, upon double-checking #255, I'm able to recreate the errors there, but there's no actual failure, just ugly-looking output. So it's possible the best solution is to just roll that back and add an FAQ stating "hey, you may get these scary errors thanks to pyyaml, but if you see 'success' at the end, don't worry". Hopefully we can get the best of both worlds though.

[bitprophet]

This is getting confusing fast, in part due to the two conflicting errors (old and new) & the difficulty in reproducing them reliably.

• The universal .whl on PyPI for 0.12.1 only seems to have the yaml3 subtree. This is what's causing all the problems for users in this thread, presumably. I can confirm that installing that wheel under Python 2 causes inv to explode.
• The same wheel for 0.12.0 has both YAML subtrees.
• When I started poking this, I thought add exclude option in setup.py #254 #255 was merged between 0.12.0 and 0.12.1, but this is not the case, it's in the changelog for 0.12.0. That means add exclude option in setup.py #254 #255 is probably not the problem.
• Building a wheel using my normal build method, under either version of Python, currently results in a .whl that has both yaml2 and yaml3.
• Therefore, somehow my build & distribution of 0.12.1 was abnormal and omitted the yaml2 tree. I have no idea how that could have happened. Perhaps it's related to how I was testing out improved build scripts at the time.

---

Additionally, trying to apply @jenisys' approach of extras_require doesn't appear to make a difference - it feels like it only applies to regular, external dependencies (the same type that are placed into install_requires) and our issue here is which sub-packages of our source tree end up in the archives. I haven't dug super deep into this yet but that's what I'm getting from the wheel docs, google results, etc.

---

So for now I am chalking this up to me screwing up the 0.12.1 wheel build somehow. I'm going to publish 0.12.2 tonight and doublecheck the distributed result for existence of both yaml trees.

When I do I will comment here and ask all that posted to please try installing 0.12.2 and report your results.

Thanks!