pyport.h FreeBSD/Mac OS X "fix" causes errors in C++ compilation

[bertjwregeer]

BPO	10910
Nosy	@brettcannon, @birkenfeld, @barry-scott, @gpshead, @ronaldoussoren, @larryhastings, @benjaminp, @ned-deily, @meadori, @mihaic, @bertjwregeer, @koobs, @DimitryAndric
Files	issue10910.txt test-boost-python.tar.gz test-boost-python.revised.tar.gz: revised simplified test case issue10910-workaround.txt issue10910-fix-versionchecks-1.diff: Correct FreeBSD version checks for enabling ctype UTF-8 workaround

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ned-deily'
closed_at = <Date 2016-08-15.07:35:14.711>
created_at = <Date 2011-01-14.21:21:37.822>
labels = ['OS-mac', 'build']
title = 'pyport.h FreeBSD/Mac OS X "fix" causes errors in C++ compilation'
updated_at = <Date 2016-08-15.07:35:14.709>
user = 'https://github.com/bertjwregeer'

bugs.python.org fields:

activity = <Date 2016-08-15.07:35:14.709>
actor = 'ned.deily'
assignee = 'ned.deily'
closed = True
closed_date = <Date 2016-08-15.07:35:14.711>
closer = 'ned.deily'
components = ['Build', 'macOS']
creation = <Date 2011-01-14.21:21:37.822>
creator = 'X-Istence'
dependencies = []
files = ['21917', '22703', '24428', '30805', '42359']
hgrepos = []
issue_num = 10910
keywords = ['patch', 'needs review']
message_count = 24.0
messages = ['126299', '130477', '133949', '134095', '135414', '140455', '140665', '140718', '140719', '152166', '152169', '152201', '152329', '152340', '152660', '163783', '192420', '192449', '192450', '262828', '272199', '272234', '272727', '272729']
nosy_count = 16.0
nosy_names = ['brett.cannon', 'georg.brandl', 'barry-scott', 'gregory.p.smith', 'ronaldoussoren', 'larry', 'benjamin.peterson', 'ned.deily', 'meador.inge', 'mihaic', 'X-Istence', 'python-dev', 'ndousis', 'Chen.Huang', 'koobs', 'dim']
pr_nums = []
priority = None
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue10910'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.6']

[bertjwregeer]

I was recently attempting to get Botan (http://botan.randombit.net) working with Python 2.6.6 on FreeBSD when it failed to compile, I filled a bug with Botan (http://bugs.randombit.net/show_bug.cgi?id=135) and first thought it was a compiler issue, and then thought it might be a Boost.Python issue.

However after further digging, see comment 3 (http://bugs.randombit.net/show_bug.cgi?id=135#c3) I figured out it had to do with the fix that went in to pyport.h for issues with isspace/toupper/et al. on FreeBSD's libc and also Mac OS X.

As can be seen in http://svn.python.org/view/python/trunk/Include/pyport.h?r1=36519&r2=36793 a change was added to override the original definitions with a new one that used the wide character support. It was modified again in http://svn.python.org/view/python/trunk/Include/pyport.h?r1=77585&r2=80178 to also include Mac OS X for http://bugs.python.org/issue7072.

The issue here is that if this file is included in any C++ code that then also includes <locale> or any of other functions that might pull in localefwd.h the defines will cause actual C++ code functions to be overwritten with invalid data, and the C++ compiler will throw errors such as:

/usr/include/c++/4.2/bits/localefwd.h:58:34: error: macro "isspace" passed 2
arguments, but takes just 1

Putting an #if 0, #endif around that block of code allows Botan's python module to cleanly compile without issues.

I do apologise that I don't have a simple C++ program that reproduces the problem.

[ndousis]

I'm upgrading from Python 2.6.5 to 2.7.1, and I'm getting the error below when compiling my code using Boost 1.45 and gcc 4.2.1 in OSX 10.6.6. The following thread describes similar symptoms related to the ordering of header files and macro definitions:
http://code.google.com/p/unladen-swallow/issues/detail?id=87#c13

The suggested patch to pyport.h (http://codereview.appspot.com/179049/patch/1/2) does not fix my problem. However, prepending

#include "Python.h"

to each source file (e.g., bend.cc below) does fix the problem. Is there a simple way to fix this without adding Python.h to every source file in my codebase?

I've installed Boost and Python from source to non-standard directories, Boost 1.45 by:
cd $(BOOST); ./bootstrap.sh --prefix=$(CURDIR)/$(BOOST) --with-python-root=$(CURDIR)/$(PYTHON_DIR) --with-python-version=2.7 --with-libraries=mpi

and Python 2.7 by:
cd $(PYTHON_SRCDIR) ; ./configure --enable-shared --prefix=$(CURDIR)/$(PYTHON_DIR)
cd $(PYTHON_SRCDIR) ; make -j $(NUMPROC) && make install

Any guidance will be greatly appreciated.

Kind regards,
Nasos

===============
...
darwin.compile.c++ bin/darwin-4.2.1/release/threading-multi/bend.o
In file included from /usr/include/c++/4.2.1/ios:47,
from /usr/include/c++/4.2.1/ostream:45,
from /usr/include/c++/4.2.1/iterator:70,
from boost/boost/next_prior.hpp:15,
from boost/boost/utility.hpp:17,
from boost/boost/python/instance_holder.hpp:10,
from boost/boost/python/object/pointer_holder.hpp:14,
from boost/boost/python/to_python_indirect.hpp:10,
from boost/boost/python/converter/arg_to_python.hpp:10,
from boost/boost/python/call.hpp:15,
from boost/boost/python/object_core.hpp:14,
from boost/boost/python/args.hpp:25,
from boost/boost/python.hpp:11,
from bend.hh:6,
from bend.cc:1:
/usr/include/c++/4.2.1/bits/localefwd.h:58:34: error: macro "isspace" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:70:34: error: macro "isupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:74:34: error: macro "islower" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:78:34: error: macro "isalpha" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:94:34: error: macro "isalnum" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:102:34: error: macro "toupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:106:34: error: macro "tolower" passed 2 arguments, but takes just 1
In file included from /usr/include/c++/4.2.1/bits/basic_ios.h:44,
from /usr/include/c++/4.2.1/ios:50,
from /usr/include/c++/4.2.1/ostream:45,
from /usr/include/c++/4.2.1/iterator:70,
from boost/boost/next_prior.hpp:15,
from boost/boost/utility.hpp:17,
from boost/boost/python/instance_holder.hpp:10,
from boost/boost/python/object/pointer_holder.hpp:14,
from boost/boost/python/to_python_indirect.hpp:10,
from boost/boost/python/converter/arg_to_python.hpp:10,
from boost/boost/python/call.hpp:15,
from boost/boost/python/object_core.hpp:14,
from boost/boost/python/args.hpp:25,
from boost/boost/python.hpp:11,
from bend.hh:6,
from bend.cc:1:
/usr/include/c++/4.2.1/bits/locale_facets.h:242:53: error: macro "toupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:271:53: error: macro "tolower" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:814:53: error: macro "toupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:847:53: error: macro "tolower" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4611:44: error: macro "isspace" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4629:44: error: macro "isupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4635:44: error: macro "islower" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4641:44: error: macro "isalpha" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4665:44: error: macro "isalnum" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4677:44: error: macro "toupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4683:44: error: macro "tolower" passed 2 arguments, but takes just 1
In file included from /usr/include/c++/4.2.1/ios:47,
from /usr/include/c++/4.2.1/ostream:45,
from /usr/include/c++/4.2.1/iterator:70,
from boost/boost/next_prior.hpp:15,
from boost/boost/utility.hpp:17,
from boost/boost/python/instance_holder.hpp:10,
from boost/boost/python/object/pointer_holder.hpp:14,
from boost/boost/python/to_python_indirect.hpp:10,
from boost/boost/python/converter/arg_to_python.hpp:10,
from boost/boost/python/call.hpp:15,
from boost/boost/python/object_core.hpp:14,
from boost/boost/python/args.hpp:25,
from boost/boost/python.hpp:11,
from bend.hh:6,
from bend.cc:1:
/usr/include/c++/4.2.1/bits/localefwd.h:58: error: ‘std::isspace’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:58: error: template declaration of ‘bool std::isspace’
/usr/include/c++/4.2.1/bits/localefwd.h:70: error: ‘std::isupper’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:70: error: template declaration of ‘bool std::isupper’
/usr/include/c++/4.2.1/bits/localefwd.h:74: error: ‘std::islower’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:74: error: template declaration of ‘bool std::islower’
/usr/include/c++/4.2.1/bits/localefwd.h:78: error: ‘std::isalpha’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:78: error: template declaration of ‘bool std::isalpha’
/usr/include/c++/4.2.1/bits/localefwd.h:94: error: ‘std::isalnum’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:94: error: template declaration of ‘bool std::isalnum’
/usr/include/c++/4.2.1/bits/localefwd.h:102: error: ‘std::toupper’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:102: error: template declaration of ‘_CharT std::toupper’
/usr/include/c++/4.2.1/bits/localefwd.h:106: error: ‘std::tolower’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:106: error: template declaration of ‘_CharT std::tolower’
In file included from /usr/include/c++/4.2.1/bits/basic_ios.h:44,
from /usr/include/c++/4.2.1/ios:50,
from /usr/include/c++/4.2.1/ostream:45,
from /usr/include/c++/4.2.1/iterator:70,
from boost/boost/next_prior.hpp:15,
from boost/boost/utility.hpp:17,
from boost/boost/python/instance_holder.hpp:10,
from boost/boost/python/object/pointer_holder.hpp:14,
from boost/boost/python/to_python_indirect.hpp:10,
from boost/boost/python/converter/arg_to_python.hpp:10,
from boost/boost/python/call.hpp:15,
from boost/boost/python/object_core.hpp:14,
from boost/boost/python/args.hpp:25,
from boost/boost/python.hpp:11,
from bend.hh:6,
from bend.cc:1:
/usr/include/c++/4.2.1/bits/locale_facets.h:227: error: ‘btowc’ is not a type
/usr/include/c++/4.2.1/bits/locale_facets.h:242: error: expected ‘;’ before ‘const’
/usr/include/c++/4.2.1/bits/locale_facets.h:255: error: expected ;' before ‘char_type’ /usr/include/c++/4.2.1/bits/locale_facets.h:256: error: ‘btowc’ is not a type /usr/include/c++/4.2.1/bits/locale_facets.h:271: error: expected ‘;’ before ‘const’ /usr/include/c++/4.2.1/bits/locale_facets.h:287: error: expected ;' before ‘char_type’
/usr/include/c++/4.2.1/bits/locale_facets.h: In member function ‘_CharT std::__ctype_abstract_base<_CharT>::towupper(int ()(_CharT)) const’:
/usr/include/c++/4.2.1/bits/locale_facets.h:228: error: ‘__c’ was not declared in this scope
/usr/include/c++/4.2.1/bits/locale_facets.h: In member function ‘_CharT std::__ctype_abstract_base<_CharT>::towlower(int ()(_CharT)) const’:
/usr/include/c++/4.2.1/bits/locale_facets.h:257: error: ‘__c’ was not declared in this scope
/usr/include/c++/4.2.1/bits/locale_facets.h: At global scope:
/usr/include/c++/4.2.1/bits/locale_facets.h:797: error: ‘btowc’ is not a type
/usr/include/c++/4.2.1/bits/locale_facets.h:814: error: expected ‘;’ before ‘const’
/usr/include/c++/4.2.1/bits/locale_facets.h:829: error: expected ;' before ‘char_type’ /usr/include/c++/4.2.1/bits/locale_facets.h:830: error: ‘btowc’ is not a type /usr/include/c++/4.2.1/bits/locale_facets.h:847: error: expected ‘;’ before ‘const’ /usr/include/c++/4.2.1/bits/locale_facets.h:866: error: expected ;' before ‘char_type’
/usr/include/c++/4.2.1/bits/locale_facets.h: In member function ‘char std::ctype::towupper(int ()(char)) const’:
/usr/include/c++/4.2.1/bits/locale_facets.h:798: error: ‘__c’ was not declared in this scope
/usr/include/c++/4.2.1/bits/locale_facets.h: In member function ‘char std::ctype::towlower(int ()(char)) const’:
/usr/include/c++/4.2.1/bits/locale_facets.h:831: error: ‘__c’ was not declared in this scope
In file included from /usr/include/c++/4.2.1/bits/basic_ios.h:44,
from /usr/include/c++/4.2.1/ios:50,
from /usr/include/c++/4.2.1/ostream:45,
from /usr/include/c++/4.2.1/iterator:70,
from boost/boost/next_prior.hpp:15,
from boost/boost/utility.hpp:17,
from boost/boost/python/instance_holder.hpp:10,
from boost/boost/python/object/pointer_holder.hpp:14,
from boost/boost/python/to_python_indirect.hpp:10,
from boost/boost/python/converter/arg_to_python.hpp:10,
from boost/boost/python/call.hpp:15,
from boost/boost/python/object_core.hpp:14,
from boost/boost/python/args.hpp:25,
from boost/boost/python.hpp:11,
from bend.hh:6,
from bend.cc:1:
/usr/include/c++/4.2.1/bits/locale_facets.h: At global scope:
/usr/include/c++/4.2.1/bits/locale_facets.h:4611: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4629: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4635: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4641: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4665: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4677: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4683: error: function definition does not declare parameters

"g++"  -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -g -dynamic -no-cpp-precomp -gdwarf-2 -fPIC -Wstrict-aliasing=2 -ffast-math -fno-strict-aliasing -DBOOST_PYTHON_MAX_ARITY=30 -D_REENTRANT  -I"boost" -I"python/include/python2.7" -c -o "bin/darwin-4.2.1/release/threading-multi/Dbend.o" "bend.cc"

...failed darwin.compile.c++ bin/darwin-4.2.1/release/threading-multi/bend.o...

[meadori]

This doesn't look like it has anything to to with the 'ctypes' library
component (http://docs.python.org/library/ctypes). So I am removing
'ctypes' from the 'Components' selection. Please correct me if I am
wrong ...