pip install klayout doesn't seem to work on M1 MacBookPro

[Crostiphex]

Hi Klayout team,

First of all thanks for making this software, you're all fantastic and I appreciate your patience. This is the first issue I've ever made, so I hope I get the situation right. Running MacBook Pro (14-inch, 2021) version 12.3 (21E230).
Using pyenv I have python 3.10.2 installed with the latest version of pip.
I want to use the klayout module, but I can't install it with pip install klayout. I've also tried to download the source distribution, went to that folder, and ran pip install . with no difference.

It seems like pip tries to make a wheel, fails, runs setup.py, and fails there too. What's clear to me is that there are some missing files and many clang errors. The terminal output is incredibly long and exceeds the maximum character limit for github's issues. Please see attached for the output:
Full Terminal Ouput.txt

I also seem to get this error when trying to install on my Intel MBP (2019), specifically the fail on setup.py bdist_wheel

[Crostiphex]

I'm sure you've looked into it, but if you have homebrew perhaps try brew install wheel and try to install it again. If failed we might have the same issue and it's more generic than I thought.

[klayoutmatthias]

Looks like yet another compiler issue:

src/db/db/dbCompoundOperation.cc:756:17: error: type 'db::CompoundRegionGeometricalBoolOperationNode::GeometricalOp' cannot be used prior to '::' because it has no members
          if (m_op == GeometricalOp::And || m_op == GeometricalOp::Not) {
                      ^

I used to build PyPI packages myself but Travis stopped their generous MacOS support. I don't have hardware to do these builds. Apple seems to adopt new clang versions much faster than my Ubuntu's clang.

In that particular case, the compiler accepts C++11 partially even if not explicitly instructed to use C++11. But then it rejects the qualified enums.

You can try adding "-std=c++11" by patching setup.py. I'd try to remove the "Darwin" branch of "if" in lines 276 and 277:

        # elif platform.system() == "Darwin":
        #     return []
        else:
            return ["-Wno-strict-aliasing",  # Avoids many "type-punned pointer" warnings
                    "-std=c++11",  # because we use unordered_map/unordered_set
                    ]

Let me know if this works, I'll patch the source package accordingly.

Matthias

[Crostiphex]

Looks like yet another compiler issue:

src/db/db/dbCompoundOperation.cc:756:17: error: type 'db::CompoundRegionGeometricalBoolOperationNode::GeometricalOp' cannot be used prior to '::' because it has no members
          if (m_op == GeometricalOp::And || m_op == GeometricalOp::Not) {
                      ^

I used to build PyPI packages myself but Travis stopped their generous MacOS support. I don't have hardware to do these builds. Apple seems to adopt new clang versions much faster than my Ubuntu's clang.

In that particular case, the compiler accepts C++11 partially even if not explicitly instructed to use C++11. But then it rejects the qualified enums.

You can try adding "-std=c++11" by patching setup.py. I'd try to remove the "Darwin" branch of "if" in lines 276 and 277:

        # elif platform.system() == "Darwin":
        #     return []
        else:
            return ["-Wno-strict-aliasing",  # Avoids many "type-punned pointer" warnings
                    "-std=c++11",  # because we use unordered_map/unordered_set
                    ]

Let me know if this works, I'll patch the source package accordingly.

Matthias

You hit the nail on its head! It worked:

 ~/Dow/klayout-0.27.8 > pip install .               
Processing /Users/me/Downloads/klayout-0.27.8
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: klayout
  Building wheel for klayout (setup.py) ... done
  Created wheel for klayout: filename=klayout-0.27.8-cp310-cp310-macosx_12_0_arm64.whl size=11607025 sha256=f04b556c9bd0b6e3128b9eadcb7e85f91c34d6f8214906fcc7a7254c02675721
  Stored in directory: /Users/me/Library/Caches/pip/wheels/8f/58/fc/18431eb5d45eb05d5faf94d91c74763691c8472953c2a8a705
Successfully built klayout
Installing collected packages: klayout
Successfully installed klayout-0.27.8

Yep, that did the trick for me also. Thanks !

[Kazzz-S]

Dear @klayoutmatthias,

Thank you very much for the patch idea.
I have tried on macOS Catalina (10.15.7) with Anaconda3, as shown below.

Regards,
Kazzz-S

(base) MacBookPro2{kazzz-s}(1)$ uname -a
Darwin MacBookPro2.local 19.6.0 Darwin Kernel Version 19.6.0: Tue Feb 15 21:39:11 PST 2022; root:xnu-6153.141.59~1/RELEASE_X86_64 x86_64


(base) MacBookPro2{kazzz-s}(2)$ pwd
/Users/kazzz-s/GitWork/klayout


(base) MacBookPro2{kazzz-s}(3)$ which python
/Applications/anaconda3/bin/python


(base) MacBookPro2{kazzz-s}(4)$ python setup.py build
:
:
:
(base) MacBookPro2{kazzz-s}(5)$ python setup.py install
:
:
:
Installed /Applications/anaconda3/lib/python3.8/site-packages/klayout-0.27.8-py3.8-macosx-10.9-x86_64.egg
Processing dependencies for klayout==0.27.8
Finished processing dependencies for klayout==0.27.8

(base) MacBookPro2{kazzz-s}(1)$ pwd
/Applications/anaconda3/lib/python3.8/site-packages


(base) MacBookPro2{kazzz-s}(2)$ ll -rt
:
drwxr-xr-x   5 kazzz-s wheel     160  3 25 18:35 klayout-0.27.8-py3.8-macosx-10.9-x86_64.egg

(base) MacBookPro2{kazzz-s}(1)$ pwd
/Users/kazzz-s/temp

(base) MacBookPro2{kazzz-s}(2)$ python
Python 3.8.2 (default, Mar 26 2020, 10:43:30)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pya
>>>


(base) MacBookPro2{kazzz-s}(3)$ ll
:
-rw-r--r--   1 kazzz-s staff 1125  3 25 18:43 sample.py


(base) MacBookPro2{kazzz-s}(4)$ python sample.py
(base) MacBookPro2{kazzz-s}(5)$ ll
:
-rw-r--r--   1 kazzz-s staff 6316  3 25 18:46 basic.gds
-rw-r--r--   1 kazzz-s staff 1125  3 25 18:43 sample.py

---

#
# "sample.py" from https://www.klayout.org/klayout-pypi/
#
import klayout.db as db

ly = db.Layout()

# sets the database unit to 1 nm
ly.dbu = 0.001

# adds a single top cell
top_cell = ly.create_cell("SAMPLE")

# creates a new layer (layer number 1, datatype 0)
layer1 = ly.layer(1, 0)

pattern = """
.#...#.#......###..#...#..###..#...#.#####
.#..#..#.....#...#.#...#.#...#.#...#...#..
.#.#...#.....#...#.#...#.#...#.#...#...#..
.##....#.....#####..#.# .#...#.#...#...#..
.#.#...#.....#...#...#. .#...#.#...#...#..
.#..#..#.....#...#...#. .#...#.#...#...#..
.#...#.#####.#...#...#. ..###...###....#..
"""

# produces pixels from the bitmap as 0.5x0.5 µm
# boxes on a 1x1 µm grid:
y = 8.0
for line in pattern.split("\n"):

  x = 0.0
  for bit in line:

    if bit == "#":
      # creates a rectangle for the "on" pixel
      rect = db.DBox(0, 0, 0.5, 0.5).moved(x, y)
      top_cell.shapes(layer1).insert(rect)

    x += 1.0

  y -= 1.0

# adds an envelope box on layer 2/0
layer2 = ly.layer(2, 0)
envelope = top_cell.dbbox().enlarged(1.0, 1.0)
top_cell.shapes(layer2).insert(envelope)

# writes the layout to GDS
ly.write("basic.gds")

---

[Added on 2022-03-26]

To build and use a pymod wheel package

OS: macOS Big Sur (11.6)
Python Environment: Anaconda3

<< Build Steps >>

% cd $HOME/GitWork/klayout  (KLayout's root directory)

% python setup.py build

% python setup.py bdist_wheel
% python setup.py bdist_egg   (optional)

% python setup.py clean --all

<< Installation >>

$ cd dist
(base) BigSur2{kazzz-s} dist (1)% pwd
/Users/kazzz-s/Gitwork/klayout/dist

(base) BigSur2{kazzz-s} dist (2)% ll
total 24660
drwxr-xr-x  4 kazzz-s staff      128  3 26 22:37 .
drwxr-xr-x 37 kazzz-s staff     1184  3 26 22:38 ..
-rw-r--r--  1 kazzz-s staff 12397786  3 26 22:37 klayout-0.27.8-cp38-cp38-macosx_10_9_x86_64.whl
-rw-r--r--  1 kazzz-s staff 12396516  3 26 22:37 klayout-0.27.8-py3.8-macosx-10.9-x86_64.egg


(base) BigSur2{kazzz-s} dist (3)% pip install klayout-0.27.8-cp38-cp38-macosx_10_9_x86_64.whl

<< Check >>

(base) BigSur2{kazzz-s} site-packages (4)% pwd
/Applications/anaconda3/lib/python3.8/site-packages

(base) BigSur2{kazzz-s} site-packages (5)% ll -rt
drwxr-xr-x  19 kazzz-s wheel     608  3 26 22:38 klayout
drwxr-xr-x   4 kazzz-s wheel     128  3 26 22:38 pya
drwxr-xr-x  10 kazzz-s wheel     320  3 26 22:38 klayout-0.27.8.dist-info


(base) BigSur2{kazzz-s} temp (6)% python
Python 3.8.2 (default, Mar 26 2020, 10:43:30)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pya
>>> help(pya)

=====================
Help on package pya:

NAME
    pya

DESCRIPTION
    # import all packages from klayout, such as klayout.db and klayout.tl
    # WARNING: doing it manually until it becomes impractical
    # TODO: We need a specification document explaining what should go into pya

PACKAGE CONTENTS


FILE
    /Applications/anaconda3/lib/python3.8/site-packages/pya/__init__.py

(END)
=====================

[klayoutmatthias]

Very good! And I see the PyPI package documentation got discovered :)

I'll establish this patch permanently for master and 0.27 branch. Thanks for you feedback!

Matthias

[thomaslima]

Hi @klayoutmatthias

I just encountered this using my non-M1 Mac (it's been a while for me). This seems to have helped me. However, on the #1045 branch, it still failed because the extensions tlcore, dbcore etc did not have the extra compilation flags in setup.py. I have added them to the pull request. Hope you don't mind.

Thanks!
Thomas

[klayoutmatthias]

Released with 0.27.9