ctypes.util.find_library fails when ldconfig/glibc not available (e.g., AIX)

[aixtools]

BPO	26439
Nosy	@vstinner, @ned-deily, @haubi, @vadmium, @aixtools, @Mariatta
PRs	bpo-26439 Fix ctypes.util.find_library failure on AIX #4507 bpo-26439: Convert %s to f-strings. #4986
Dependencies	bpo-22636: avoid using a shell in ctypes.util: replace os.popen with subprocess
Files	python.Lib.ctypes.160309.patch python.Lib.ctypes.160317.patch python.Lib.ctypes.160317.patch: Regenerated with Mercurial python.Lib.ctypes.160504.patch: New patches to init.py and util.py as described in message aixutil.py: proposed ctypes.aixutil Library code to support ctypes.cdll and ctypes.util python.Lib.ctypes.160504.patch: Regenerated, including aixutil python.Lib.ctypes.160509.patch aixutil.py: aixutil.py for Lib/ctypes PythonX.Lib.ctypes.aixutil.py.160510: Lib/ctypes/aixutil.py addition Python2.Lib.ctypes.160510.patch: Python2/Lib/ctypes pathces Python2.Modules._ctypes.160510.patch: Python2/Modules/_ctypes/_ctype.c patch to export RTLD_MEMBER and RTLD_NOW Python3.Lib.ctypes.160510.patch: Python3/Lib/ctypes patches Python3.Modules._ctypes.160510.patch: Python3/Modules/_ctypes/_ctype.c patch to export RTLD_MEMBER and RTLD_NOW Python3.issue26439.160511.patch: new patch for Python3 - specific for this issue Python2.issue26439.160511.patch: new patch for Python2 - specific for this issue aixutil.py: aixutil.py for Lib/ctypes - updated Python2.Lib.ctypes.160531.patch: Python2/Lib/ctypes patch _aixutil.py python.Lib.ctypes.160608.patch: diff -ru results for Python-2.7.11/Lib/ctypes _aixutil.py: revised _aixutil.py for review Python2.Lib.ctypes.160611.patch: revised ctypes/Lib patches based on Python 2.7 Python2.Lib.ctypes.160611.patch: Regenerated for Rietveld Python3.6.Lib.ctypes.160823.patch: Add support for AIX platform to Lib/ctypes Python3.6.Modules._ctypes.160823.patch: Add RTLD_MEMBER to Modules/_ctypes Python3.6.ctypes.160823.patch: Combined, regenerated Python3.6.Lib.ctypes.160904.patch: patch compared to prevous patch (dated 160823) Python3.6.Lib.ctypes.160926.patch: delta from Python3.6.0b1 to provide find_library() support compareable to Linux aix-library.161001.patch aix-modules.161004.patch: update to _ctypes.c to add RTLD_MEMBER aix-library.161004.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-07-18.20:57:26.917>
created_at = <Date 2016-02-25.15:10:23.154>
labels = ['ctypes', 'type-bug', '3.7']
title = 'ctypes.util.find_library fails when ldconfig/glibc not available (e.g., AIX)'
updated_at = <Date 2018-07-18.20:57:26.916>
user = 'https://github.com/aixtools'

bugs.python.org fields:

activity = <Date 2018-07-18.20:57:26.916>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2018-07-18.20:57:26.917>
closer = 'vstinner'
components = ['ctypes']
creation = <Date 2016-02-25.15:10:23.154>
creator = 'Michael.Felt'
dependencies = ['22636']
files = ['42094', '42193', '42634', '42712', '42713', '42776', '42789', '42790', '42801', '42802', '42803', '42804', '42805', '42811', '42812', '43073', '43074', '43311', '43312', '43425', '43426', '43479', '44205', '44206', '44237', '44366', '44831', '44902', '44962', '44963']
hgrepos = ['359']
issue_num = 26439
keywords = ['patch']
message_count = 90.0
messages = ['260861', '260862', '260863', '260864', '260885', '260890', '260982', '260999', '261010', '261012', '261013', '261392', '261932', '261934', '264150', '264391', '264392', '264405', '264406', '264451', '264458', '264514', '264544', '264806', '264807', '265112', '265115', '265194', '265195', '265253', '265278', '265291', '265302', '265304', '265305', '265306', '265435', '266132', '266211', '266527', '266718', '266783', '266784', '267254', '267420', '267900', '267902', '267903', '268083', '268203', '268214', '268355', '268401', '268696', '268697', '268699', '268884', '268885', '271742', '271816', '273503', '273505', '273762', '273977', '274335', '274354', '274374', '277428', '277491', '277801', '278066', '278083', '278086', '278087', '278092', '278675', '285395', '285396', '307421', '308636', '308638', '308746', '308759', '308773', '308860', '308948', '308954', '315311', '321916', '321917']
nosy_count = 9.0
nosy_names = ['vstinner', 'ned.deily', 'haubi', 'python-dev', 'martin.panter', 'David.Edelsohn', 'Michael.Felt', 'aixtools@gmail.com', 'Mariatta']
pr_nums = ['4507', '4986']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue26439'
versions = ['Python 3.7']

[aixtools]

I have successful enough with python 2.7.10 (for building cloud-init), including it finding openssl libraries during the installation od setuptools (before installing pip).

I have also been able to assemble saltstack - BUT - salt-master and salt-minion fail to start because ctypes.util.find_library() always returns 'None'.
======== EXCERPT ====
File "/opt/lib/python2.7/site-packages/salt/crypt.py", line 37, in <module>
import salt.utils.rsax931
File "/opt/lib/python2.7/site-packages/salt/utils/rsax931.py", line 69, in <module>
libcrypto = _init_libcrypto()
File "/opt/lib/python2.7/site-packages/salt/utils/rsax931.py", line 47, in _init_libcrypto
libcrypto = _load_libcrypto()
File "/opt/lib/python2.7/site-packages/salt/utils/rsax931.py", line 40, in _load_libcrypto
raise OSError('Cannot locate OpenSSL libcrypto')
OSError: Cannot locate OpenSSL libcrypto
=======
I built python using the IBM compiler, and my images do not have /sbin/ldconfig installed so the assumption that /sbin/ldconfig is always installed is "a bug".

in the util.py file the code reached is:

+219 else:
+220
+221 def _findSoname_ldconfig(name):
+222 import struct
+223 if struct.calcsize('l') == 4:
+224 machine = os.uname()[4] + '-32'
+225 else:
+226 machine = os.uname()[4] + '-64'
+227 mach_map = {
+228 'x86_64-64': 'libc6,x86-64',
+229 'ppc64-64': 'libc6,64bit',
+230 'sparc64-64': 'libc6,64bit',
+231 's390x-64': 'libc6,64bit',
+232 'ia64-64': 'libc6,IA-64',
+233 }
+234 abi_type = mach_map.get(machine, 'libc6')
+235
+236 # XXX assuming GLIBC's ldconfig (with option -p)
+237 expr = r'\s+(lib%s\.[^\\s]+)\s+\(%s' % (re.escape(name), abi_type)
+238 f = os.popen('/sbin/ldconfig -p 2>/dev/null')
+239 try:
+240 data = f.read()
+241 finally:
+242 f.close()
+243 res = re.search(expr, data)
+244 if not res:
+245 return None
+246 return res.group(1)
+247
+248 def find_library(name):
+249 return _findSoname_ldconfig(name) or _get_soname(_findLib_gcc(name))
+250

(I have not researched _get_soname or _findLib_gcc but neither of these "feel right" as AIX, by default, does not end library (archives) with .so (archives end with .a and archive members frequently end with .so)

That this is/has not been reported more frequently may be because python programmers are avoiding ctypes - when portability is essential.

I hope that, just as for Solaris - where an alternate program is used - that AIX can have a block:
# if os.name == "posix" and sys.platform.startswith("aix"):

so that if ldconfig is not available the command /usr/bin/dump could be used instead, and/or search LIBPATH and/or (when not empty) and/or ldd (for programs). Ideally, /sbin/ldconfig would not be need at all!

dump output:

root@x064:[/data/prj/gnu/bashRC1-4.4]dump -Xany -H /opt/bin/python

/opt/bin/python:

                    \*\*\*Loader Section***
                  Loader Header Information

VERSION# #SYMtableENT #RELOCent LENidSTR
0x00000001 0x000005ac 0x000035e3 0x0000006e

#IMPfilID OFFidSTR LENstrTBL OFFstrTBL
0x00000005 0x00030ee4 0x00006772 0x00030f52

                    \*\*\*Import File Strings***

INDEX PATH BASE MEMBER
0 /usr/vac/lib:/usr/lib:/lib
1 libc.a shr.o
2 libpthreads.a shr_xpg5.o
3 libpthreads.a shr_comm.o
4 libdl.a shr.o

root@x064:[/usr/bin]dump -Xany -H /usr/lib/libcrypto.a

/usr/lib/libcrypto.a[libcrypto.so]:

                    \*\*\*Loader Section***
                  Loader Header Information

VERSION# #SYMtableENT #RELOCent LENidSTR
0x00000001 0x00000ff9 0x0000498a 0x00000038

#IMPfilID OFFidSTR LENstrTBL OFFstrTBL
0x00000003 0x0004f1f0 0x00014636 0x0004f228

                    \*\*\*Import File Strings***

INDEX PATH BASE MEMBER
0 /usr/lib:/lib
1 libc.a shr.o
2 libpthreads.a shr_xpg5.o

/usr/lib/libcrypto.a[libcrypto.so.0.9.8]:

                    \*\*\*Loader Section***
                  Loader Header Information

VERSION# #SYMtableENT #RELOCent LENidSTR
0x00000001 0x00000c4e 0x00004312 0x00000038

#IMPfilID OFFidSTR LENstrTBL OFFstrTBL
0x00000003 0x00044c48 0x0000f236 0x00044c80

                    \*\*\*Import File Strings***

INDEX PATH BASE MEMBER
0 /usr/lib:/lib
1 libc.a shr.o
2 libpthreads.a shr_xpg5.o

/usr/lib/libcrypto.a[libcrypto.so.1.0.0]:

                    \*\*\*Loader Section***
                  Loader Header Information

VERSION# #SYMtableENT #RELOCent LENidSTR
0x00000001 0x00000ff9 0x0000498a 0x00000038

#IMPfilID OFFidSTR LENstrTBL OFFstrTBL
0x00000003 0x0004f1f0 0x00014636 0x0004f228

                    \*\*\*Import File Strings***

INDEX PATH BASE MEMBER
0 /usr/lib:/lib
1 libc.a shr.o
2 libpthreads.a shr_xpg5.o

/usr/lib/libcrypto.a[libcrypto64.so]:

                    \*\*\*Loader Section***
                  Loader Header Information

VERSION# #SYMtableENT #RELOCent LENidSTR
0x00000001 0x00000ff2 0x00004987 0x0000003e

#IMPfilID OFFidSTR LENstrTBL OFFstrTBL
0x00000003 0x00061758 0x00014e03 0x00061796

                    \*\*\*Import File Strings***

INDEX PATH BASE MEMBER
0 /usr/lib:/lib
1 libc.a shr_64.o
2 libpthreads.a shr_xpg5_64.o

/usr/lib/libcrypto.a[libcrypto64.so.0.9.8]:

                    \*\*\*Loader Section***
                  Loader Header Information

VERSION# #SYMtableENT #RELOCent LENidSTR
0x00000001 0x00000c47 0x0000430d 0x0000003e

#IMPfilID OFFidSTR LENstrTBL OFFstrTBL
0x00000003 0x000557b0 0x0000f9e7 0x000557ee

                    \*\*\*Import File Strings***

INDEX PATH BASE MEMBER
0 /usr/lib:/lib
1 libc.a shr_64.o
2 libpthreads.a shr_xpg5_64.o

/usr/lib/libcrypto.a[libcrypto64.so.1.0.0]:

                    \*\*\*Loader Section***
                  Loader Header Information

VERSION# #SYMtableENT #RELOCent LENidSTR
0x00000001 0x00000ff2 0x00004987 0x0000003e

#IMPfilID OFFidSTR LENstrTBL OFFstrTBL
0x00000003 0x00061758 0x00014e03 0x00061796

                    \*\*\*Import File Strings***

INDEX PATH BASE MEMBER
0 /usr/lib:/lib
1 libc.a shr_64.o
2 libpthreads.a shr_xpg5_64.o
root@x064:[/usr/bin]
root@x064:[/usr/bin]echo $?
0

ldd outpath:
root@x064:[/usr/bin]ldd /opt/bin/python
/opt/bin/python needs:
/usr/lib/libc.a(shr.o)
/usr/lib/libpthreads.a(shr_xpg5.o)
/usr/lib/libpthreads.a(shr_comm.o)
/usr/lib/libdl.a(shr.o)
/unix
/usr/lib/libcrypt.a(shr.o)

root@x064:[/usr/bin]ldd /usr/lib/libcrypto.a
ldd: /usr/lib/libcrypto.a: File is an archive.
root@x064:[/usr/bin]echo $?
2

[aixtools]

p.s. On a debian (on POWER) system, the function is working, but the test seems a bit broken as well, i.e., cdll.LoadLibrary("libm.so") is not working even though

    if os.name == "posix":
        # find and load_version
        print find_library("m")
        print find_library("c")
        print find_library("bz2")

has successfully printed.
root@ipv4:/home/michael# python -m ctypes.util
libm.so.6
libc.so.6
libbz2.so.1.0
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/lib/python2.7/ctypes/util.py", line 287, in <module>
    test()
  File "/usr/lib/python2.7/ctypes/util.py", line 282, in test
    print cdll.LoadLibrary("libm.so")
  File "/usr/lib/python2.7/ctypes/__init__.py", line 443, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libm.so: cannot open shared object file: No such file or directory
root@ipv4:/home/michael# uname -a
Linux ipv4.rootvg.net 3.16.0-4-powerpc64 #1 SMP Debian 3.16.7-ckt9-3 (2015-04-23) ppc64 GNU/Linux
root@ipv4:/home/michael# ldconfig -p | grep libm.so
        libm.so.6 (libc6, OS ABI: Linux 2.6.32) => /lib/powerpc-linux-gnu/libm.so.6
root@ipv4:/home/michael# ls -l /lib/powerpc-linux-gnu/libm.so.6
lrwxrwxrwx 1 root root 12 Apr 15  2015 /lib/powerpc-linux-gnu/libm.so.6 -> libm-2.19.so
root@ipv4:/home/michael# ls -l /lib/powerpc-linux-gnu/libm-2.19.so
-rw-r--r-- 1 root root 743784 Apr 15  2015 /lib/powerpc-linux-gnu/libm-2.19.so

[aixtools]

Further testing...

I added an extremely simple hack in util.py so that a archive (AIX library) name got returned.

Also in this case - the print cdll.LoadLibrary("libc.a") fails.

+74 if os.name == "posix" and sys.platform == "darwin":
+75 from ctypes.macholib.dyld import dyld_find as _dyld_find
+76 def find_library(name):
+77 possible = ['lib%s.dylib' % name,
+78 '%s.dylib' % name,
+79 '%s.framework/%s' % (name, name)]
+80 for name in possible:
+81 try:
+82 return _dyld_find(name)
+83 except ValueError:
+84 continue
+85 return None
+86
+87 if os.name == "posix" and sys.platform.startswith("aix"):
+88 def find_library(name):
+89 expr = "lib" + name + ".a"
+90 return expr
+91
+92 elif os.name == "posix":
+93 # Andreas Degert's find functions, using gcc, /sbin/ldconfig, objdump
+94 import re, tempfile, errno
+95
+96 def _findLib_gcc(name):

+275 else:
+276 print cdll.LoadLibrary("libc.a")
+277 print cdll.LoadLibrary("libm.so")
+278 print cdll.LoadLibrary("libcrypt.so")

root@x064:[/tmp]python -m ctypes.util
libm.a
libc.a
libbz2.a
Traceback (most recent call last):
  File "/opt/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/opt/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/opt/lib/python2.7/ctypes/util.py", line 282, in <module>
    test()
  File "/opt/lib/python2.7/ctypes/util.py", line 276, in test
    print cdll.LoadLibrary("libc.a")
  File "/opt/lib/python2.7/ctypes/__init__.py", line 443, in LoadLibrary
    return self._dlltype(name)
  File "/opt/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError:        0509-022 Cannot load module /usr/lib/libc.a.
        0509-103   The module has an invalid magic number.

So, what needs to be returned so that cdll.LoadLibrary could use that result? (e.g., I know that the default libm.a has no shared members on AIX 5.3 TL7 - but libc.a does (shr.o and more, but not all!).

[aixtools]

Last message (back to debian, and minor changes to learn expected behavior)

        if sys.platform == "darwin":
            print cdll.LoadLibrary("libm.dylib")
            print cdll.LoadLibrary("libcrypto.dylib")
            print cdll.LoadLibrary("libSystem.dylib")
            print cdll.LoadLibrary("System.framework/System")
        else:
            print cdll.LoadLibrary("libm.so.6")
#            print cdll.LoadLibrary("libcrypt.so")
            print find_library("crypt")
            x = find_library("crypt")
            print cdll.LoadLibrary(x)

returns:
root@ipv4:/home/michael# python -m ctypes.util
libm.so.6
libc.so.6
libbz2.so.1.0
<CDLL 'libm.so.6', handle f7e65528 at f7b51fb0>
libcrypt.so.1
<CDLL 'libcrypt.so.1', handle 106a1878 at f7b51fb0>

[aixtools]

FYI: getting objdump is not too hard...

root@x064:[/data/prj/gnu/binutils-2.25.1]v/null /usr/lib/libcrypto.a <
In archive /usr/lib/libcrypto.a:

libcrypto.so: file format aixcoff-rs6000

libcrypto.so.0.9.8: file format aixcoff-rs6000

libcrypto.so.1.0.0: file format aixcoff-rs6000

libcrypto64.so: file format aix5coff64-rs6000

libcrypto64.so.0.9.8: file format aix5coff64-rs6000

libcrypto64.so.1.0.0: file format aix5coff64-rs6000

But ldconfig (as a command within glibc) will be a real headache - and I would home unnecessary.

In closing, I hope the AIX command /usr/bin/dump will be adequate as an alternative of objdump.

Where I am still a bit lost is with the "open", i.e. _dlopen(). Suggestions/hints for debugging are welcome.

[aixtools]

The _dlopen call in __init__.py I have been able to fix (hack) with the following:

root@x064:[/data/prj/aixtools/python/python-2.7.10/Lib/ctypes]diff -u __init__.py /opt/lib/python2.7/ctypes/__init__.py
--- __init__.py 2015-05-23 16:09:01 +0000
+++ /opt/lib/python2.7/ctypes/__init__.py       2016-02-26 08:40:19 +0000
@@ -11,6 +11,7 @@
 from _ctypes import _Pointer
 from _ctypes import CFuncPtr as _CFuncPtr
 from _ctypes import __version__ as _ctypes_version
+# from _ctypes import RTLD_LOCAL, RTLD_GLOBAL, RTLD_NOW ## fails
 from _ctypes import RTLD_LOCAL, RTLD_GLOBAL
 from _ctypes import ArgumentError
 
@@ -32,6 +33,11 @@
     if int(_os.uname()[2].split('.')[0]) < 8:
         DEFAULT_MODE = RTLD_GLOBAL
 
+if _os.name == "posix" and _sys.platform.startswith("aix"):
+        RTLD_NOW    = 0x00000002
+        RTLD_MEMBER = 0x00040000
+        DEFAULT_MODE |= (RTLD_NOW | RTLD_MEMBER)
+
 from _ctypes import FUNCFLAG_CDECL as _FUNCFLAG_CDECL, \
      FUNCFLAG_PYTHONAPI as _FUNCFLAG_PYTHONAPI, \
      FUNCFLAG_USE_ERRNO as _FUNCFLAG_USE_ERRNO, \

I have an additional hack in util.py so that, e.g.,
find_Library("crypto")

returns:
'/usr/lib/libcrypto.a(libcrypto.so)'

When that is passed to dlopen (plus RTLD_MEMBER) the dlopen succeeds.

<CDLL '/usr/lib/libcrypto.a/(libcrypto.so)', handle 6 at 30146670>

With some help in util.py, to do some sensible searching, being able to establish if in 32 or 64-bit mode, etc. somehing nice can be made here (imho).

p.s. can any verify whether this is limited to python 2.7? Or should python 3.X be added as well?

[DavidEdelsohn]

ctypes util.py "simply" needs support for AIX. There already is special support for Windows, Darwin, BSDs, Solaris. Is the question about the technical details for equivalent functionality on AIX or about adding a stanza to Lib/ctpyes/util.py?

[aixtools]

it was, partly, about the technical details - but I feel I have found the key bits - /full/path/libNAME.a(libNAME.so) + dlflags RTLD_NOW + RTLD_MEMBER for "native" AIX support.

Additionally, a patch should not break what might be working for some (via google I read of suggestions for other languages, e.g., java, where .so members are extracted or installed side-by-side with the .a archive) - as the IBM run-rime loader also accepts/looks for both .a and .so. And/or for when /sbin/ldconfig is available. And when available, is this used as last case, or preferred?

Lastly, we cannot assume we will know the name of the member based on the name of the library. In many cases, e.g., libiconv.a the IBM names are shr.o, shr4.o and shr4_64.o - and these are the only member names unless GNU libiconv.a has been added (and then libconv.a also contains libiconv.so.2 (so version support is also desired!)) in both 32 and 64 bit mode.

One technical detail I have not been able to discover yet: how to determine whether in 32-bit or 64-bit mode. This will be important for libraries that have shr4.o and shr4_64.o as the members that need to be dlopened.

Lastly: about adding the "stanza": python is a new language for me. Any assistance with a patch - to keep it properly 'pythonized'

In short, I shall continue my studies/learning. Assistance from anyone wiser than me is welcome!

[DavidEdelsohn]

AIX traditionally used member names like shr.o or shr<version>.o or shr<aix_release>.o insider the archive, with _64 designating a 64 bit object when there is a naming collision.

GNU libtool defaults to the SO name and version number insider the archive.

AIX objects (and shared objects) contain a bit in the header that specifies 32 bit or 64 bit. Both 32 bit and 64 bit objects are intended to be archived together. The linker only processes objects of the correct mode.

AIX shared objects contain a bit that specifies if the object may be used at link-edit time or only should be used for loading. This is controlled by the AIX strip -e/-E option (yes, I know, strange place to hide that option).

This combination of features allows all of the libraries to be placed in a single /usr/lib directory and all of the objects to be collected into a single archive, avoiding /usr/lib64 and explosion of shared objects and symbolic links clutter. Various packages have created /usr/local/lib64 anyway using Linux/Solaris/SVR4-style naming.

[aixtools]

The following demonstrates the basics.

cdll.LoadLibrary will load a .so file if it can be located (you will have to believe me as I forgot to include the test in this last batch)

Further, when given the AIX format for loading a member of an archive
(e.g., libc.a(shr.o) or libcrypto.a(libcrypto.so), the dlload occurs as expected.

in ctypes/utils.py
This looks in the standard directories (as the example, but not complete, see below) to demonstrate how to respond.
Note: it is also looking for a .so file, and if it finds one, it returns the .so name, otherwise it looks for a .a file - and when it finds it, just appends the libNAME.so in the correct format.

What is not done, and probably different from ldconfig behavior!

1. The complete path is not returned (so cdll.LoadLibrary will still follow default search - which I hope includes, read uses, the default for the python executable

2. The member name is not verified for existence (e.g., libconv.a(libiconv.so) is not expected to find libiconv.a(libiconv.so.2)

3. Since member verification is not being performed, shr.o, shr4.o, shr*_64.o

And many many thanks for the strip -e/-E info. Hard to find!

root@x064:[/data/prj/aixtools/src]diff -u python-2.7.11/Lib/ctypes/__init__.py /opt/lib/python2.7/ctypes/__init__.py
--- python-2.7.11/Lib/ctypes/__init__.py        2015-12-05 19:46:56 +0000
+++ /opt/lib/python2.7/ctypes/__init__.py       2016-02-29 17:36:43 +0000
@@ -11,6 +11,7 @@
 from _ctypes import _Pointer
 from _ctypes import CFuncPtr as _CFuncPtr
 from _ctypes import __version__ as _ctypes_version
+# from _ctypes import RTLD_LOCAL, RTLD_GLOBAL, RTLD_NOW ## fails
 from _ctypes import RTLD_LOCAL, RTLD_GLOBAL
 from _ctypes import ArgumentError
 
@@ -356,6 +357,14 @@
         if use_last_error:
             flags |= _FUNCFLAG_USE_LASTERROR
 
+        if _sys.platform.startswith("aix"):
+            RTLD_NOW    = 0x00000002
+            RTLD_MEMBER = 0x00040000
+            mode |= RTLD_NOW
+            if name is not None:
+                if name[-1] == ')':
+                       mode |= RTLD_MEMBER
+
         class _FuncPtr(_CFuncPtr):
             _flags_ = flags
             _restype_ = self._func_restype_


root@x064:[/data/prj/aixtools/src]diff -u python-2.7.11/Lib/ctypes/util.py /opt/lib/python2.7/ctypes/util.py
--- python-2.7.11/Lib/ctypes/util.py    2015-12-05 19:46:56 +0000
+++ /opt/lib/python2.7/ctypes/util.py   2016-02-29 17:42:41 +0000
@@ -84,6 +84,23 @@
                 continue
         return None
 
+if os.name == "posix" and sys.platform.startswith("aix"):
+        def _findLib_aix(name):
+            paths='/usr/lib:/lib:/usr/lpp/xlC/lib'
+            for dir in paths.split(":"):
+                shlib = os.path.join(dir, "lib%s.so" % name)
+                if os.path.exists(shlib):
+                    return shlib
+                libfile = os.path.join(dir, "lib%s.a" % name)
+                if os.path.exists(libfile):
+                    shlib = "lib%s.a(lib%s.so)" % (name, name)
+                    return shlib
+
+            return None
+
+        def find_library(name):
+            return _findLib_aix(name)
+
 elif os.name == "posix":
     # Andreas Degert's find functions, using gcc, /sbin/ldconfig, objdump
     import re, tempfile, errno
@@ -267,7 +284,16 @@
             print cdll.LoadLibrary("libcrypto.dylib")
             print cdll.LoadLibrary("libSystem.dylib")
             print cdll.LoadLibrary("System.framework/System")
+        elif sys.platform.startswith("aix"):
+            print find_library("crypto")
+            x = find_library("crypto")
+            print cdll.LoadLibrary(x)
+            print cdll.LoadLibrary("libcrypto.a(libcrypto.so)")
+            print cdll.LoadLibrary("libc.a(shr.o)")
+            print find_library("crypt")
+            print cdll.LoadLibrary("libcrypt.a(shr.o)")
         else:
+            print cdll.LoadLibrary("libc.a")
             print cdll.LoadLibrary("libm.so")
             print cdll.LoadLibrary("libcrypt.so")
             print find_library("crypt")

p.s. Anyone know what flag to set in vi so that tabs are never inserted and/or spaces converted to tabs. Python seems to hate tab indents.

[aixtools]

Oops: forgot the example output:

root@x064:[/data/prj/aixtools/src]python -m ctypes.util
libm.a(libm.so)
libc.a(libc.so)
libbz2.a(libbz2.so)
libcrypto.a(libcrypto.so)
<CDLL 'libcrypto.a(libcrypto.so)', handle 6 at 30147690>
<CDLL 'libcrypto.a(libcrypto.so)', handle 7 at 30147690>
<CDLL 'libc.a(shr.o)', handle 8 at 30147690>
libcrypt.a(libcrypt.so)
<CDLL 'libcrypt.a(shr.o)', handle 9 at 30147690>

[aixtools]

The patch works when installed on top of pre-compiled version (e.g., copy the two files to /opt/lib/python2.7/ctypes/

but there seems to be a nasty side-effect when trying to build.

make completes, but make install fails during a compile step.

Please assist with howto see how/why ctypes/util is impacting this part.

And - for my first adventure into python please provide the needed (expected) feedback.

Michael

make install DESTDIR=/var/aixtools/aixtools/python/2.7.11.2 > .buildaix/install.out

Listing /var/aixtools/aixtools/python/2.7.11.2/opt/lib/python2.7/xml/sax ...
Compiling /var/aixtools/aixtools/python/2.7.11.2/opt/lib/python2.7/xml/sax/init.py ...
Compiling /var/aixtools/aixtools/python/2.7.11.2/opt/lib/python2.7/xml/sax/_exceptions.py ...
Compiling /var/aixtools/aixtools/python/2.7.11.2/opt/lib/python2.7/xml/sax/expatreader.py ...
Compiling /var/aixtools/aixtools/python/2.7.11.2/opt/lib/python2.7/xml/sax/handler.py ...
Compiling /var/aixtools/aixtools/python/2.7.11.2/opt/lib/python2.7/xml/sax/saxutils.py ...
Compiling /var/aixtools/aixtools/python/2.7.11.2/opt/lib/python2.7/xml/sax/xmlreader.py ...
Compiling /var/aixtools/aixtools/python/2.7.11.2/opt/lib/python2.7/xmllib.py ...
Compiling /var/aixtools/aixtools/python/2.7.11.2/opt/lib/python2.7/xmlrpclib.py ...
Compiling /var/aixtools/aixtools/python/2.7.11.2/opt/lib/python2.7/zipfile.py ...

So, zipfile.py seems to exit non-zero. :(