Mac OSX, Python: import klayout.dbcore, missing libpng16.16.dylib

[lukasc-ubc]

@Kazzz-S

The KLayout Python package is referencing a library "libpng16.16.dylib" that is not installed on my system. I understand that it can be installed by brew. Perhaps this library could be bundled as part of the "pip install" klayout package?

Thank you

(base) iMac-Hickup:anaconda3 lukasc$ pip install klayout
Collecting klayout
  Downloading klayout-0.28.3-cp39-cp39-macosx_10_9_x86_64.whl (16.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.7/16.7 MB 10.4 MB/s eta 0:00:00
Installing collected packages: klayout
Successfully installed klayout-0.28.3
(base) iMac-Hickup:anaconda3 lukasc$ python 
Python 3.9.13 (main, Aug 25 2022, 18:29:29) 
[Clang 12.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
im>>> import klayout.dbcore
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: dlopen(/Users/lukasc/opt/anaconda3/lib/python3.9/site-packages/klayout/dbcore.cpython-39-darwin.so, 2): Library not loaded: /usr/local/opt/libpng/lib/libpng16.16.dylib
  Referenced from: /Users/lukasc/opt/anaconda3/lib/python3.9/site-packages/klayout/lib_tl.cpython-39-darwin.so
  Reason: image not found

[Kazzz-S]

Hi @lukasc-ubc,

I think libpng*.dylib is independent of pip install klayout.

(base) iMac-Hickup:anaconda3 lukasc$ pip install klayout
Collecting klayout
  Downloading klayout-0.28.3-cp39-cp39-macosx_10_9_x86_64.whl (16.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.7/16.7 MB 10.4 MB/s eta 0:00:00
Installing collected packages: klayout
Successfully installed klayout-0.28.3
(base) iMac-Hickup:anaconda3 lukasc$ python
Python 3.9.13 (main, Aug 25 2022, 18:29:29)
[Clang 12.0.0 ] :: Anaconda, Inc. on darwin
                   ^^^^^^^^^^^^^^
Type "help", "copyright", "credits" or "license" for more information.
im>>> import klayout.dbcore
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: dlopen(/Users/lukasc/opt/anaconda3/lib/python3.9/site-packages/klayout/dbcore.cpython-39-darwin.so,
 2): Library not loaded: /usr/local/opt/libpng/lib/libpng16.16.dylib  <===== (1)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Referenced from: /Users/lukasc/opt/anaconda3/lib/python3.9/site-packages/klayout/lib_tl.cpython-39-darwin.so
  Reason: image not found

(1) It looks like Homebrew's libpng16.16.dylib is being loaded. But your current environment is Anaconda3.

---

In my Anaconda3 environment on Monterey...

(base) MacBookPro2{sekigawa} ~ (1)% which pip
/Applications/anaconda3/bin/pip

(base) MacBookPro2{sekigawa} ~ (2)% pip list | grep -E '(klay|png)'
klayout                       0.28.3 <===== (2)

(base) MacBookPro2{sekigawa} ~ (3)% which python3
/Applications/anaconda3/bin/python3

(base) MacBookPro2{sekigawa} ~ (4)% python3
Python 3.9.15 (main, Nov  4 2022, 11:11:31)
[Clang 12.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import klayout.dbcore
    ^^^^^^^^^^^^^^^^^^^^^  No errors
>>> help(klayout.dbcore)

Help on module klayout.dbcore in klayout:

NAME
    klayout.dbcore - KLayout core module 'db'

CLASSES
    builtins.object
        Box
        Cell
        CellInstArray
        CellMapping
        CompoundRegionOperationNode
        Connectivity
        CplxTrans
        DBox
        DCellInstArray
        DCplxTrans
        DEdge
        DEdgePair
        DPath
        DPoint
        DPolygon
        DSimplePolygon
        DText
        DTrans
        DVector
        DeepShapeStore
        DeviceAbstract
        DeviceAbstractRef
        DeviceClass
            DeviceClassBJT3Transistor
                DeviceClassBJT4Transistor
            DeviceClassCapacitor
                DeviceClassCapacitorWithBulk
            DeviceClassDiode
            DeviceClassInductor
            DeviceClassMOS3Transistor
                DeviceClassMOS4Transistor
            DeviceClassResistor
                DeviceClassResistorWithBulk
        DeviceClassFactory
        DeviceExtractorBase
            DeviceExtractorBJT3Transistor
                DeviceExtractorBJT4Transistor
:
: 
: 

---

(2) My "pymod" is homemade, not from PyPI. However, I believe there is no difference.
The LW*.dmg (published in the KLayout site, such as "LW-klayout-0.28.3-macOS-Monterey-1-qt5Ana3-Rana3Pana3.dmg") contains a *.whl like:

(base) MacBookPro2{sekigawa} pymod-dist (1)% pwd
/Applications/klayout-ana32.app/Contents/pymod-dist
              ^^^^^^^^^^^^^^^^^          ^^^^^^^^^^
              I've renamed the application bundle.

(base) MacBookPro2{sekigawa} pymod-dist (2)% ll
total 27272
drwxr-xr-x 4 sekigawa admin      128 Jan 15 06:27 .
drwxr-xr-x 9 sekigawa admin      288 Jan 15 06:27 ..
-rwxr-xr-x 1 sekigawa admin 13961592 Jan 15 06:27 klayout-0.28.3-cp39-cp39-macosx_10_9_x86_64.whl <===== home-made 'pymod'
-rwxr-xr-x 1 sekigawa admin 13962423 Jan 15 06:27 klayout-0.28.3-py3.9-macosx-10.9-x86_64.egg

(base) MacBookPro2{sekigawa} pymod-dist (3)% which pip3
/Applications/anaconda3/bin/pip3

(base) MacBookPro2{sekigawa} pymod-dist (4)% pip3 install klayout-0.28.3-cp39-cp39-macosx_10_9_x86_64.whl -U
Processing ./klayout-0.28.3-cp39-cp39-macosx_10_9_x86_64.whl
klayout is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel.

Let's see the site-packages.

(base) MacBookPro2{sekigawa} klayout (1)% pwd
/Applications/anaconda3/lib/python3.9/site-packages/klayout

(base) MacBookPro2{sekigawa} klayout (2)% ll
total 48156
drwxr-xr-x  33 sekigawa wheel     1056 Jan 15 08:03 .
drwxr-xr-x 698 sekigawa wheel    22336 Jan 25 07:56 ..
-rw-r--r--   1 sekigawa wheel       34 Jan 15 08:03 __init__.py
drwxr-xr-x   3 sekigawa wheel       96 Jan 15 08:03 __pycache__
drwxr-xr-x   5 sekigawa wheel      160 Jan 15 08:03 db
drwxr-xr-x  10 sekigawa wheel      320 Jan 15 08:03 db_plugins
-rwxr-xr-x   1 sekigawa wheel    35928 Jan 15 08:03 dbcore.cpython-39-darwin.so
-rw-r--r--   1 sekigawa wheel  2376826 Jan 15 08:03 dbcore.pyi
drwxr-xr-x   4 sekigawa wheel      128 Jan 15 08:03 lay
-rwxr-xr-x   1 sekigawa wheel    35992 Jan 15 08:03 laycore.cpython-39-darwin.so
-rw-r--r--   1 sekigawa wheel   585462 Jan 15 08:03 laycore.pyi
drwxr-xr-x   4 sekigawa wheel      128 Jan 15 08:03 lib
-rwxr-xr-x   1 sekigawa wheel   567912 Jan 15 08:03 lib_ant.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel 27942736 Jan 15 08:03 lib_db.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel   960264 Jan 15 08:03 lib_edt.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel   816152 Jan 15 08:03 lib_gsi.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel   772784 Jan 15 08:03 lib_img.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel 10772720 Jan 15 08:03 lib_laybasic.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel   182976 Jan 15 08:03 lib_layview.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel   285256 Jan 15 08:03 lib_lib.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel   441536 Jan 15 08:03 lib_lym.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel  1386456 Jan 15 08:03 lib_pya.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel    38840 Jan 15 08:03 lib_rba.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel  1018968 Jan 15 08:03 lib_rdb.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel   750576 Jan 15 08:03 lib_tl.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel    35736 Jan 15 08:03 libcore.cpython-39-darwin.so
-rw-r--r--   1 sekigawa wheel      151 Jan 15 08:03 libcore.pyi
drwxr-xr-x   4 sekigawa wheel      128 Jan 15 08:03 rdb
-rwxr-xr-x   1 sekigawa wheel    35648 Jan 15 08:03 rdbcore.cpython-39-darwin.so
-rw-r--r--   1 sekigawa wheel    71673 Jan 15 08:03 rdbcore.pyi
drwxr-xr-x   4 sekigawa wheel      128 Jan 15 08:03 tl
-rwxr-xr-x   1 sekigawa wheel    35528 Jan 15 08:03 tlcore.cpython-39-darwin.so
-rw-r--r--   1 sekigawa wheel   118783 Jan 15 08:03 tlcore.pyi

(base) MacBookPro2{sekigawa} klayout (3)% otool -L lib_tl.cpython-39-darwin.so | grep png
  @rpath/libpng16.16.dylib (compatibility version 54.0.0, current version 54.0.0) <===== (3)

---

(3) This "libpng16.16.dylib" is Anaconda3's, which was installed by default.

(base) MacBookPro2{sekigawa} lib (1)% pwd
/Applications/anaconda3/lib

(base) MacBookPro2{sekigawa} lib (2)% ll | grep png
lrwxr-xr-x    1 sekigawa wheel        10 Dec 30 23:26 libpng.a -> libpng16.a
lrwxr-xr-x    1 sekigawa wheel        17 Dec 30 23:26 libpng.dylib -> libpng16.16.dylib
-rwxrwxr-x    1 sekigawa wheel    221836 Dec 30 23:26 libpng16.16.dylib <===== (3)
-rw-rw-r--    2 sekigawa wheel    305384 Apr 24  2019 libpng16.a
lrwxr-xr-x    1 sekigawa wheel        17 Dec 30 23:26 libpng16.dylib -> libpng16.16.dylib
-rwxrwxr-x    1 sekigawa wheel    247616 Dec 30 23:26 libzopflipng.1.0.3.dylib
lrwxr-xr-x    1 sekigawa wheel        24 Dec 30 23:26 libzopflipng.1.dylib -> libzopflipng.1.0.3.dylib
lrwxr-xr-x    1 sekigawa wheel        24 Dec 30 23:26 libzopflipng.dylib -> libzopflipng.1.0.3.dylib

(base) MacBookPro2{sekigawa} lib (3)% otool -L libpng16.16.dylib
libpng16.16.dylib:
  @rpath/libpng16.16.dylib (compatibility version 54.0.0, current version 54.0.0)
  @rpath/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
  /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)

(base) MacBookPro2{sekigawa} ~ (4)% conda list | grep png
libpng                    1.6.37               ha441bb4_0  

Regards,
Kazzz-S

[Kazzz-S]

In the Homebrew environment...

MacBookPro2{sekigawa} ~ (1)% which pip3
/usr/local/opt/python@3.9/bin/pip3

MacBookPro2{sekigawa} ~ (2)% pip3 list | grep -E '(klay|png)'
klayout         0.28.3

MacBookPro2{sekigawa} ~ (3)% which python3
/usr/local/opt/python@3.9/bin/python3

MacBookPro2{sekigawa} ~ (4)% python3
Python 3.9.16 (main, Dec  7 2022, 10:15:43)
[Clang 14.0.0 (clang-1400.0.29.202)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import klayout.dbcore
    ^^^^^^^^^^^^^^^^^^^^^ No errors
>>> help(klayout.dbcore)

Help on module klayout.dbcore in klayout:

NAME
    klayout.dbcore - KLayout core module 'db'

CLASSES
    builtins.object
        Box
        Cell
        CellInstArray
        CellMapping
        CompoundRegionOperationNode
        Connectivity
        CplxTrans
        DBox
        DCellInstArray
        DCplxTrans
        DEdge
        DEdgePair
        DPath
        DPoint
        DPolygon
        DSimplePolygon
        DText
        DTrans
        DVector
        DeepShapeStore
        DeviceAbstract
        DeviceAbstractRef
        DeviceClass
            DeviceClassBJT3Transistor
                DeviceClassBJT4Transistor
            DeviceClassCapacitor
                DeviceClassCapacitorWithBulk
            DeviceClassDiode
            DeviceClassInductor
            DeviceClassMOS3Transistor
                DeviceClassMOS4Transistor
            DeviceClassResistor
                DeviceClassResistorWithBulk
        DeviceClassFactory
        DeviceExtractorBase
            DeviceExtractorBJT3Transistor
                DeviceExtractorBJT4Transistor
:
:
:

---

MacBookPro2{sekigawa} klayout (1)% pwd
/usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/Current/lib/python3.9/site-packages/klayout

MacBookPro2{sekigawa} klayout (2)% ll
total 60692
drwxr-xr-x 33 sekigawa admin     1056 Jan 15 08:04 .
drwxr-xr-x 46 sekigawa admin     1472 Jan 15 08:04 ..
-rw-r--r--  1 sekigawa admin       34 Jan 15 08:04 __init__.py
drwxr-xr-x  3 sekigawa admin       96 Jan 15 08:04 __pycache__
drwxr-xr-x  5 sekigawa admin      160 Jan 15 08:04 db
drwxr-xr-x 10 sekigawa admin      320 Jan 15 08:04 db_plugins
-rwxr-xr-x  1 sekigawa admin    36648 Jan 15 08:04 dbcore.cpython-39-darwin.so
-rw-r--r--  1 sekigawa admin  2376826 Jan 15 08:04 dbcore.pyi
drwxr-xr-x  4 sekigawa admin      128 Jan 15 08:04 lay
-rwxr-xr-x  1 sekigawa admin    36632 Jan 15 08:04 laycore.cpython-39-darwin.so
-rw-r--r--  1 sekigawa admin   585462 Jan 15 08:04 laycore.pyi
drwxr-xr-x  4 sekigawa admin      128 Jan 15 08:04 lib
-rwxr-xr-x  1 sekigawa admin   822368 Jan 15 08:04 lib_ant.cpython-39-darwin.so
-rwxr-xr-x  1 sekigawa admin 36299944 Jan 15 08:04 lib_db.cpython-39-darwin.so
-rwxr-xr-x  1 sekigawa admin  1227800 Jan 15 08:04 lib_edt.cpython-39-darwin.so
-rwxr-xr-x  1 sekigawa admin  1255296 Jan 15 08:04 lib_gsi.cpython-39-darwin.so
-rwxr-xr-x  1 sekigawa admin  1086184 Jan 15 08:04 lib_img.cpython-39-darwin.so
-rwxr-xr-x  1 sekigawa admin 12106800 Jan 15 08:04 lib_laybasic.cpython-39-darwin.so
-rwxr-xr-x  1 sekigawa admin   255128 Jan 15 08:04 lib_layview.cpython-39-darwin.so
-rwxr-xr-x  1 sekigawa admin   372080 Jan 15 08:04 lib_lib.cpython-39-darwin.so
-rwxr-xr-x  1 sekigawa admin   635912 Jan 15 08:04 lib_lym.cpython-39-darwin.so
-rwxr-xr-x  1 sekigawa admin  2068888 Jan 15 08:04 lib_pya.cpython-39-darwin.so
-rwxr-xr-x  1 sekigawa admin    43272 Jan 15 08:04 lib_rba.cpython-39-darwin.so
-rwxr-xr-x  1 sekigawa admin  1523696 Jan 15 08:04 lib_rdb.cpython-39-darwin.so
-rwxr-xr-x  1 sekigawa admin  1073824 Jan 15 08:04 lib_tl.cpython-39-darwin.so
-rwxr-xr-x  1 sekigawa admin    36440 Jan 15 08:04 libcore.cpython-39-darwin.so
-rw-r--r--  1 sekigawa admin      151 Jan 15 08:04 libcore.pyi
drwxr-xr-x  4 sekigawa admin      128 Jan 15 08:04 rdb
-rwxr-xr-x  1 sekigawa admin    36304 Jan 15 08:04 rdbcore.cpython-39-darwin.so
-rw-r--r--  1 sekigawa admin    71673 Jan 15 08:04 rdbcore.pyi
drwxr-xr-x  4 sekigawa admin      128 Jan 15 08:04 tl
-rwxr-xr-x  1 sekigawa admin    36080 Jan 15 08:04 tlcore.cpython-39-darwin.so
-rw-r--r--  1 sekigawa admin   118783 Jan 15 08:04 tlcore.pyi

MacBookPro2{sekigawa} klayout (3)% otool -L lib_tl.cpython-39-darwin.so | grep png
    /usr/local/opt/libpng/lib/libpng16.16.dylib (compatibility version 56.0.0, current version 56.0.0)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

MacBookPro2{sekigawa} lib (4)% pwd
/usr/local/opt/libpng/lib

MacBookPro2{sekigawa} lib (5)% ll
total 436
drwxr-xr-x  8 sekigawa admin    256 Nov 21 06:53 .
drwxr-xr-x 13 sekigawa admin    416 Nov 25 06:45 ..
lrwxr-xr-x  1 sekigawa admin     10 Nov 21 06:53 libpng.a -> libpng16.a
lrwxr-xr-x  1 sekigawa admin     14 Nov 21 06:53 libpng.dylib -> libpng16.dylib
-rw-r--r--  1 sekigawa admin 189368 Nov 25 06:45 libpng16.16.dylib
-rw-r--r--  1 sekigawa admin 253560 Nov 21 06:53 libpng16.a
lrwxr-xr-x  1 sekigawa admin     17 Nov 21 06:53 libpng16.dylib -> libpng16.16.dylib
drwxr-xr-x  4 sekigawa admin    128 Nov 25 06:45 pkgconfig

MacBookPro2{sekigawa} ~ (6)% brew info libpng
==> libpng: stable 1.6.39 (bottled), HEAD
Library for manipulating PNG images
http://www.libpng.org/pub/png/libpng.html
/usr/local/Cellar/libpng/1.6.39 (27 files, 1.3MB) *
  Poured from bottle on 2022-11-25 at 06:45:02
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/libpng.rb
License: libpng-2.0
==> Options
--HEAD
	Install HEAD version
==> Analytics
install: 479,081 (30 days), 1,574,660 (90 days), 3,638,819 (365 days)
install-on-request: 11,749 (30 days), 44,637 (90 days), 92,773 (365 days)
build-error: 22 (30 days)

[Kazzz-S]

In the MacPorts environment...

MacBookPro2{sekigawa} ~ (1)% which pip3
/opt/local/bin/pip3

MacBookPro2{sekigawa} ~ (2)% pip3 list | grep -E '(klay|png)'
klayout           0.28.3

MacBookPro2{sekigawa} ~ (3)% which python3
/opt/local/bin/python3

MacBookPro2{sekigawa} ~ (4)% python3
Python 3.9.16 (main, Dec  7 2022, 02:41:07)
[Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import klayout.dbcore
>>> help(klayout.dbcore)
    ^^^^^^^^^^^^^^^^^^^^^ No errors
>>> help(klayout.dbcore)

Help on module klayout.dbcore in klayout:

NAME
    klayout.dbcore - KLayout core module 'db'

CLASSES
    builtins.object
        Box
        Cell
        CellInstArray
        CellMapping
        CompoundRegionOperationNode
        Connectivity
        CplxTrans
        DBox
        DCellInstArray
        DCplxTrans
        DEdge
        DEdgePair
        DPath
        DPoint
        DPolygon
        DSimplePolygon
        DText
        DTrans
        DVector
        DeepShapeStore
        DeviceAbstract
        DeviceAbstractRef
        DeviceClass
            DeviceClassBJT3Transistor
                DeviceClassBJT4Transistor
            DeviceClassCapacitor
                DeviceClassCapacitorWithBulk
            DeviceClassDiode
            DeviceClassInductor
            DeviceClassMOS3Transistor
                DeviceClassMOS4Transistor
            DeviceClassResistor
                DeviceClassResistorWithBulk
        DeviceClassFactory
        DeviceExtractorBase
            DeviceExtractorBJT3Transistor
                DeviceExtractorBJT4Transistor
:
:
:

---

MacBookPro2{sekigawa} klayout (1)% pwd
/opt/local/Library/Frameworks/Python.framework/Versions/Current/lib/python3.9/site-packages/klayout

MacBookPro2{sekigawa} klayout (2)% ll
total 51496
drwxr-xr-x 33 root wheel     1056 Jan 15 08:04 .
drwxr-xr-x 53 root wheel     1696 Jan 15 08:04 ..
-rw-r--r--  1 root wheel       34 Jan 15 08:04 __init__.py
drwxr-xr-x  3 root wheel       96 Jan 15 08:04 __pycache__
drwxr-xr-x  5 root wheel      160 Jan 15 08:04 db
drwxr-xr-x 10 root wheel      320 Jan 15 08:04 db_plugins
-rwxr-xr-x  1 root wheel    36808 Jan 15 08:04 dbcore.cpython-39-darwin.so
-rw-r--r--  1 root wheel  2376826 Jan 15 08:04 dbcore.pyi
drwxr-xr-x  4 root wheel      128 Jan 15 08:04 lay
-rwxr-xr-x  1 root wheel    36632 Jan 15 08:04 laycore.cpython-39-darwin.so
-rw-r--r--  1 root wheel   585462 Jan 15 08:04 laycore.pyi
drwxr-xr-x  4 root wheel      128 Jan 15 08:04 lib
-rwxr-xr-x  1 root wheel   724384 Jan 15 08:04 lib_ant.cpython-39-darwin.so
-rwxr-xr-x  1 root wheel 28863104 Jan 15 08:04 lib_db.cpython-39-darwin.so
-rwxr-xr-x  1 root wheel  1098504 Jan 15 08:04 lib_edt.cpython-39-darwin.so
-rwxr-xr-x  1 root wheel  1107688 Jan 15 08:04 lib_gsi.cpython-39-darwin.so
-rwxr-xr-x  1 root wheel   922744 Jan 15 08:04 lib_img.cpython-39-darwin.so
-rwxr-xr-x  1 root wheel 11181016 Jan 15 08:04 lib_laybasic.cpython-39-darwin.so
-rwxr-xr-x  1 root wheel   228768 Jan 15 08:04 lib_layview.cpython-39-darwin.so
-rwxr-xr-x  1 root wheel   319816 Jan 15 08:04 lib_lib.cpython-39-darwin.so
-rwxr-xr-x  1 root wheel   560840 Jan 15 08:04 lib_lym.cpython-39-darwin.so
-rwxr-xr-x  1 root wheel  2088888 Jan 15 08:04 lib_pya.cpython-39-darwin.so
-rwxr-xr-x  1 root wheel    43384 Jan 15 08:04 lib_rba.cpython-39-darwin.so
-rwxr-xr-x  1 root wheel  1331664 Jan 15 08:04 lib_rdb.cpython-39-darwin.so
-rwxr-xr-x  1 root wheel   885072 Jan 15 08:04 lib_tl.cpython-39-darwin.so
-rwxr-xr-x  1 root wheel    36440 Jan 15 08:04 libcore.cpython-39-darwin.so
-rw-r--r--  1 root wheel      151 Jan 15 08:04 libcore.pyi
drwxr-xr-x  4 root wheel      128 Jan 15 08:04 rdb
-rwxr-xr-x  1 root wheel    36304 Jan 15 08:04 rdbcore.cpython-39-darwin.so
-rw-r--r--  1 root wheel    71673 Jan 15 08:04 rdbcore.pyi
drwxr-xr-x  4 root wheel      128 Jan 15 08:04 tl
-rwxr-xr-x  1 root wheel    36088 Jan 15 08:04 tlcore.cpython-39-darwin.so
-rw-r--r--  1 root wheel   118783 Jan 15 08:04 tlcore.pyi

MacBookPro2{sekigawa} klayout (3)% otool -L lib_tl.cpython-39-darwin.so | grep png
    /opt/local/lib/libpng16.16.dylib (compatibility version 56.0.0, current version 56.0.0)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

MacBookPro2{sekigawa} lib (4)% pwd
/opt/local/lib

MacBookPro2{sekigawa} lib (5)% ll | grep png
lrwxr-xr-x   1 root     wheel        10 Dec  7 01:26 libpng.a -> libpng16.a
lrwxr-xr-x   1 root     wheel        14 Dec  7 01:26 libpng.dylib -> libpng16.dylib
-rwxr-xr-x   1 root     wheel    189464 Dec  7 01:26 libpng16.16.dylib
-rw-r--r--   1 root     wheel    251760 Dec  7 01:26 libpng16.a
lrwxr-xr-x   1 root     wheel        17 Dec  7 01:26 libpng16.dylib -> libpng16.16.dylib

MacBookPro2{sekigawa} ~ (6)% port installed | grep png
  libpng @1.6.39_0 (active)

[Kazzz-S]

Oh, I understand!

Again in the Anaconda3 environment

(base) MacBookPro2{sekigawa} ~ (1)% which pip3
/Applications/anaconda3/bin/pip3

(base) MacBookPro2{sekigawa} ~ (2)% pip3 uninstall klayout <===== Uninstall the existing "pymod"
Found existing installation: klayout 0.28.3
Uninstalling klayout-0.28.3:
  Would remove:
    /Applications/anaconda3/lib/python3.9/site-packages/klayout-0.28.3.dist-info/*
    /Applications/anaconda3/lib/python3.9/site-packages/klayout/*
    /Applications/anaconda3/lib/python3.9/site-packages/pya/*
Proceed (Y/n)? Y
  Successfully uninstalled klayout-0.28.3

(base) MacBookPro2{sekigawa} ~ (3)% pip3 list | grep kl
cloudpickle                   2.0.0
mkl-fft                       1.3.1
mkl-random                    1.2.2
mkl-service                   2.4.0
pickleshare                   0.7.5

(base) MacBookPro2{sekigawa} ~ (4)% pip3 install klayout  <===== Reinstall "pymod" from PyPI
Collecting klayout
  Downloading klayout-0.28.3-cp39-cp39-macosx_10_9_x86_64.whl (16.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.7/16.7 MB 1.7 MB/s eta 0:00:00
Installing collected packages: klayout
Successfully installed klayout-0.28.3

(base) MacBookPro2{sekigawa} ~ (5)% pip3 list | grep kl
cloudpickle                   2.0.0
klayout                       0.28.3 <===== OK
mkl-fft                       1.3.1
mkl-random                    1.2.2
mkl-service                   2.4.0
pickleshare                   0.7.5

(base) MacBookPro2{sekigawa} ~ (6)% which python3
/Applications/anaconda3/bin/python3

(base) MacBookPro2{sekigawa} ~ (7)% python3
Python 3.9.15 (main, Nov  4 2022, 11:11:31)
[Clang 12.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import klayout.dbcore
    ^^^^^^^^^^^^^^^^^^^^^ No errors
>>> help(klayout.dbcore)

Help on module klayout.dbcore in klayout:

NAME
    klayout.dbcore - KLayout core module 'db'

CLASSES
    builtins.object
        Box
        Cell
        CellInstArray
        CellMapping
        CompoundRegionOperationNode
        Connectivity
        CplxTrans
        DBox
        DCellInstArray
        DCplxTrans
        DEdge
        DEdgePair
        DPath
        DPoint
        DPolygon
        DSimplePolygon
        DText
        DTrans
        DVector
        DeepShapeStore
        DeviceAbstract
        DeviceAbstractRef
        DeviceClass
            DeviceClassBJT3Transistor
                DeviceClassBJT4Transistor
            DeviceClassCapacitor
                DeviceClassCapacitorWithBulk
            DeviceClassDiode
            DeviceClassInductor
            DeviceClassMOS3Transistor
                DeviceClassMOS4Transistor
            DeviceClassResistor
                DeviceClassResistorWithBulk
        DeviceClassFactory
        DeviceExtractorBase
            DeviceExtractorBJT3Transistor
                DeviceExtractorBJT4Transistor
:
:
:

---

(base) MacBookPro2{sekigawa} klayout (1)% pwd
/Applications/anaconda3/lib/python3.9/site-packages/klayout

(base) MacBookPro2{sekigawa} klayout (2)% ll
total 64528
drwxr-xr-x  33 sekigawa wheel     1056 Jan 26 21:59 .
drwxr-xr-x 698 sekigawa wheel    22336 Jan 26 21:59 ..
-rw-r--r--   1 sekigawa wheel       34 Jan 26 21:59 __init__.py
drwxr-xr-x   3 sekigawa wheel       96 Jan 26 21:59 __pycache__
drwxr-xr-x   5 sekigawa wheel      160 Jan 26 21:59 db
drwxr-xr-x  10 sekigawa wheel      320 Jan 26 21:59 db_plugins
-rwxr-xr-x   1 sekigawa wheel    37416 Jan 26 21:59 dbcore.cpython-39-darwin.so
-rw-r--r--   1 sekigawa wheel  2376826 Jan 26 21:59 dbcore.pyi
drwxr-xr-x   4 sekigawa wheel      128 Jan 26 21:59 lay
-rwxr-xr-x   1 sekigawa wheel    37208 Jan 26 21:59 laycore.cpython-39-darwin.so
-rw-r--r--   1 sekigawa wheel   585462 Jan 26 21:59 laycore.pyi
drwxr-xr-x   4 sekigawa wheel      128 Jan 26 21:59 lib
-rwxr-xr-x   1 sekigawa wheel   816520 Jan 26 21:59 lib_ant.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel 40088912 Jan 26 21:59 lib_db.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel  1249784 Jan 26 21:59 lib_edt.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel  1252608 Jan 26 21:59 lib_gsi.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel  1093376 Jan 26 21:59 lib_img.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel 12203128 Jan 26 21:59 lib_laybasic.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel   245984 Jan 26 21:59 lib_layview.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel   383624 Jan 26 21:59 lib_lib.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel   634560 Jan 26 21:59 lib_lym.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel  2075936 Jan 26 21:59 lib_pya.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel    43968 Jan 26 21:59 lib_rba.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel  1522320 Jan 26 21:59 lib_rdb.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel  1078096 Jan 26 21:59 lib_tl.cpython-39-darwin.so
-rwxr-xr-x   1 sekigawa wheel    37000 Jan 26 21:59 libcore.cpython-39-darwin.so
-rw-r--r--   1 sekigawa wheel      151 Jan 26 21:59 libcore.pyi
drwxr-xr-x   4 sekigawa wheel      128 Jan 26 21:59 rdb
-rwxr-xr-x   1 sekigawa wheel    36864 Jan 26 21:59 rdbcore.cpython-39-darwin.so
-rw-r--r--   1 sekigawa wheel    71673 Jan 26 21:59 rdbcore.pyi
drwxr-xr-x   4 sekigawa wheel      128 Jan 26 21:59 tl
-rwxr-xr-x   1 sekigawa wheel    36640 Jan 26 21:59 tlcore.cpython-39-darwin.so
-rw-r--r--   1 sekigawa wheel   118783 Jan 26 21:59 tlcore.pyi

(base) MacBookPro2{sekigawa} klayout (3)% otool -L lib_tl.cpython-39-darwin.so | grep png
  /usr/local/opt/libpng/lib/libpng16.16.dylib (compatibility version 56.0.0, current version 56.0.0)
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
             !Homebrew's library!

It seems PyPI's "pymod" was built with Homebrew.
Because I have the Homebrew environment, import klayout.dbcore was successful.
However, the twisted environment is dirty.

(2) My "pymod" is homemade, not from PyPI. However, I believe there is no difference.

The above statement was incorrect.

[Kazzz-S]

If you use "LW-klayout-0.28.3-macOS-Monterey-1-qt5Ana3-Rana3Pana3.dmg", reinstall the homemade "pymod" as follows.

(base) MacBookPro2{sekigawa} pymod-dist (1)% pwd
/Applications/klayout.app/Contents/pymod-dist

(base) MacBookPro2{sekigawa} pymod-dist (2)% ll
total 27272
drwxr-xr-x 4 sekigawa admin      128 Jan 15 06:27 .
drwxr-xr-x 9 sekigawa admin      288 Jan 15 06:27 ..
-rwxr-xr-x 1 sekigawa admin 13961592 Jan 15 06:27 klayout-0.28.3-cp39-cp39-macosx_10_9_x86_64.whl
-rwxr-xr-x 1 sekigawa admin 13962423 Jan 15 06:27 klayout-0.28.3-py3.9-macosx-10.9-x86_64.egg

(base) MacBookPro2{sekigawa} pymod-dist (3)% pip3 install klayout-0.28.3-cp39-cp39-macosx_10_9_x86_64.whl \
--force-reinstall

Processing ./klayout-0.28.3-cp39-cp39-macosx_10_9_x86_64.whl
Installing collected packages: klayout
  Attempting uninstall: klayout
    Found existing installation: klayout 0.28.3
    Uninstalling klayout-0.28.3:
      Successfully uninstalled klayout-0.28.3
Successfully installed klayout-0.28.3

I recalled how I find the <png.h> and <libpng*> to build "pymod" in the three environments.

[klayoutmatthias]

I have no real idea how to package additional files.

The PyPI packages are built using github actions. The builds depend on pypa/cibuildwheel (https://github.com/pypa/cibuildwheel) which itself employs auditwheel or delocate which (to my understanding) should add all missing files to the package.

So either the problem is that something is broken there or the github hosts already have libpng and delocate does not have a reason to add these files.

Either way, I'm lost. Maybe someone else can help.

Matthias

[lukasc-ubc]

Hi folks,

Thanks for your thoughts. I realized that the HW and LW packages are for the GUI build, and pip import is a different beast all together that uses the local Python installation (rather than shipping with Python like the HW version).

I was able to solve it by installing libpng using brew. In my case, it took more effort, including upgrading brew and installaing libpng:

git -C /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core fetch --unshallow
brew upgrade
brew install libpng

thank you

[jan-david-fischbach]

Hi folks, same issue here. However my homebrew installs libpng here

/opt/homebrew/lib/libpng16.16.dylib

meaning, that I cannot rectify the issue the way that @lukasc-ubc proposed

[jan-david-fischbach]

Installing the homemade pymod works. It is however a major inconvenience to have to do that on every install e.g. in virtual environments.

[klayoutmatthias]

Bad news, but I can only emphasize, that I have no clue how to fix it. This needs someone with a deeper knowledge about packaging wheels. The details are hidden far too deep in the guts of these CI "helpers".

I'm personally tired of digging into that stuff. I can do so, but that would put an end to KLayout feature development. No time left.

[sebastian-goeldi]

This seems to be something related to where homebrew install libpng on m1. We are currently also running into this problem (I am running linux but colleagues are using M1 or newer). I requested an M1 from my IT and should get it soon:tm:. On suggestion that I was thinking about and that probably would make sense is to instead of relying on the dynamic library, compile libpng into the executable (probably makes it bigger, but might be less annoying).

Homebrew/discussions#2625

@Hsuan-Tung could you try this as long as we don't have a testing machine yet?

[Hsuan-Tung]

When using conda environment to install, it works for me. I am using M1 mac. Here is what I did,

1. conda create -n your_venv_name python=3.10
2. conda activate your_venv_name
3. conda install libpng
4. conda install libpng-devel-cos7-aarch64 (may not be necessary)
5. export LDFLAGS="-L $(libpng-config --libdir)"
6. cd klayout (go to source klayout repo folder)
7. python -m pip install .

But directly doing brew install libpng and pip install klayout won't work for me. Always getting this error, src/tl/tl/tlPixelBuffer.cc:28:12: fatal error: 'png.h' file not found # include <png.h> ^~~~~~~

[lukasc-ubc]

Hi folks,

thank you for the help. I have completely removed Anaconda from my system, and installed Python using brew and pyenv, and the problem is fixed.

iMac-Hickup:matplotlib_example_node_pack-main lukasc$ pip install klayout
Requirement already satisfied: klayout in /usr/local/lib/python3.10/site-packages (0.28.3)
iMac-Hickup:matplotlib_example_node_pack-main lukasc$ python
Python 3.10.9 (main, Dec 15 2022, 18:20:40) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import klayout.dbcore
>>>

[klayoutmatthias]

That is interesting. Sorry for not being more constructive.

Do you feel able to summarize the analysis for me? I am not really familiar with the ways HomeBrew, Anaconda and the system libraries interact. I'm simply a Linux guy.

Thanks,

Matthias