Fix function cast warning in thread_pthread.h

[siddhesh]

BPO	33015
Nosy	@gpshead, @pitrou, @vstinner, @benjaminp, @skrah, @serhiy-storchaka, @zooba, @siddhesh, @izbyshev, @miss-islington
PRs	bpo-33015: Add a wrapper for thread function in PyThread_start_new_thread #6008 bpo-33015: Fix func cast warn in PyThread_start_new_thread() #10057 [3.7] bpo-33015: Fix UB in pthread PyThread_start_new_thread (GH-6008) #10821 [3.6] bpo-33015: Fix UB in pthread PyThread_start_new_thread (GH-6008) #10822 [2.7] bpo-33015: Fix UB in pthread PyThread_start_new_thread (GH-6008) #10823 [2.7] bpo-35368: Make PyMem_Malloc() thread-safe in debug mode #10828 [2.7] bpo-33015: Use malloc() in PyThread_start_new_thread() #10829
Files	func_cast.c

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-11-30.23:11:09.461>
created_at = <Date 2018-03-06.18:56:13.815>
labels = ['3.8', 'build', '3.7']
title = 'Fix function cast warning in thread_pthread.h'
updated_at = <Date 2018-11-30.23:11:09.460>
user = 'https://github.com/siddhesh'

bugs.python.org fields:

activity = <Date 2018-11-30.23:11:09.460>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2018-11-30.23:11:09.461>
closer = 'vstinner'
components = ['Build']
creation = <Date 2018-03-06.18:56:13.815>
creator = 'siddhesh'
dependencies = []
files = ['47888']
hgrepos = []
issue_num = 33015
keywords = ['patch']
message_count = 32.0
messages = ['313357', '316332', '316348', '316415', '316639', '316709', '328298', '328300', '328301', '328302', '328308', '328310', '328313', '328325', '328387', '328388', '328409', '328415', '328417', '328838', '330791', '330794', '330798', '330799', '330800', '330806', '330807', '330810', '330829', '330830', '330832', '330835']
nosy_count = 10.0
nosy_names = ['gregory.p.smith', 'pitrou', 'vstinner', 'benjamin.peterson', 'skrah', 'serhiy.storchaka', 'steve.dower', 'siddhesh', 'izbyshev', 'miss-islington']
pr_nums = ['6008', '10057', '10821', '10822', '10823', '10828', '10829']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue33015'
versions = ['Python 2.7', 'Python 3.6', 'Python 3.7', 'Python 3.8']

[siddhesh]

The PyThread_start_new_thread function takes a void ()(void *) as the function argument, which does not match with the pthread_create callback which has type void *()(void *). I've got a fix for this that adds a wrapper function of the right type that subsequently calls the function passed to PyThread_start_new_thread.

PR coming up.

[zooba]

Can't we just fix the cast? We shouldn't have to do heap allocations for this.

[siddhesh]

gcc8.1 throws this warning irrespective of the cast since converting function pointers may result in undefined behaviour unless it is cast back to its original type.

[zooba]

In this case, we know the behaviour is okay and the warning is wrong. We should suppress the warning around the offending line, rather than adding significant code that may introduce genuine bugs.

[siddhesh]

Actually it is not; the parameter passed to Pythread_start_new_thread has a different type (void ()(void *)) from what's accepted (and executed by) pthread_create (void *()(void *)). That is undefined behaviour.

[zooba]

Ah okay, fair enough. Knowing that, I'll take another look at the PR. I'd really like to simplify this as much as possible to avoid risk of regression.

[vstinner]

gcc8.1 throws this warning irrespective of the cast since converting function pointers may result in undefined behaviour unless it is cast back to its original type.

Do you have a reference explaining this issue? I dislike adding a memory allocation to call a function just to fix a compiler warning. From my point of view, at the end, the function pointer is just an integer. I don't understand how an additional memory allocation solves the issue.

[vstinner]

"Fix function cast warning in thread_pthread.h"

Can you please paste the warning?

[vstinner]

func_cast.c: C program reproducing the issue. Using an additional (void*) cast, it's possible to workaround the cast warning.

/* Test GCC 8.1 -Wcast-function-type for https://bugs.python.org/issue33015
*

• Compiled on Linux with:
• gcc x.c -o x -Wall -Wextra -lpthread
• Workaround the cast:
• gcc x.c -o x -Wall -Wextra -lpthread -D UGLY_CAST
  */

/* No result value */
typedef void (*python_callback) (void *);

/* Result type: "void*" (untyped pointer) */
typedef void* (*pthread_callback) (void *);

int test_cast(python_callback func)
{
    ...
#ifdef UGLY_CAST
    pthread_callback func2 = (pthread_callback)(void *)func;
#else
    pthread_callback func2 = (pthread_callback)func;
#endif
    ...
}

[vstinner]

The GCC warning is:

func_cast.c:34:30: warning: cast between incompatible function types from 'python_callback' {aka 'void ()(void *)'} to 'void * ()(void *)' [-Wcast-function-type]
pthread_callback func2 = (pthread_callback)func;
^

[vstinner]

I wrote a different fix for the compiler warning using a temporary cast to "void*": PR 10057.

[pitrou]

Right, so one PR is a real fix, the other PR is a workaround (avoids the warning without fixing the underlying problem).

The underlying problem is: if a platform has incompatible ABIs for the two function types, casting one function type to another may produce bugs (memory corruptions, crashes, etc). We can however decide to consider those platforms unlikely, as the current code has been working for years or decades.

It would be nice to have an opinion from C specialists.

[zooba]

Unfortunately, this isn't really a safe cast, as we're going from void to non-void return value.

On x86 with current calling conventions, this is okay, since the return value is in a register that does not change or require cleanup by the caller. However, I wouldn't want to assume that all future calling conventions on other architectures would also permit this - returning a pointer value on the stack or in some way that requires cleanup is entirely possible, and is the sort of problem that would likely only be detectable by this warning or very careful memory measurements (or possibly a very confusing crash due to invalid stack modifications).

It's also possible that returning an invalid pointer could cause a compiler to one day invoke its undefined behavior clause. Even though *we* don't use the return value, it still gets returned somewhere.

I'm not thrilled about the memory allocation here either, but there isn't really much of an option in my opinion.