Unloading a Rust dylib with TLS used segfaults on OSX

[alexcrichton]

Example code

The problem here is that we register a TLS destructor via _tlv_atexit when TLS is referenced the first time after it is used (e.g. when the dylib's function is called), but then when dlclose happens the function isn't actually there and a fault happens when the thread exits and tries to run its destructors.

I'm not entirely sure how we might handle this, perhaps there's a way to compile dylibs such that the TLS access is OK? Perhaps we should hook an "unload" event and deregister (e.g. leak) TLS destructors? Either way seems like a good thing to track!

[ranma42]

_dyld_register_func_for_remove_image might be the hook we need.
Manpage available at https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man3/dyld.3.html

[emoon]

cc @emoon

[emoon]

Has there been any progress on this issue? It can be worked around but not releasing the lib but it would be nice to have a proper solution.

[noeleont]

I had the same issue, one workaround was loading the lib with RTLD_NODELETE.

[solarretrace]

I think I'm having the same issue? But the RTLD_NODELETE option doesn't seem to help. It's also probably not appropriate for my use-case, which requires unloading and reloading the library repeatedly.

Stack looks like this:

Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0000000115ab5cb0

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 libsystem_malloc.dylib 0x00007fff8e8c5059 free + 58
1 dyld 0x00007fff6542cec1 ImageLoaderMachOCompressed::~ImageLoaderMachOCompressed() + 33
2 dyld 0x00007fff6541afc4 dyld::garbageCollectImages() + 831
3 dyld 0x00007fff65422428 dlclose + 134
4 libdyld.dylib 0x00007fff91143808 dlclose + 61
5 brightlab 0x000000010c54cfdb libloading::os::unix::{{impl}}::drop::{{closure}} + 43 (mod.rs:38)
6 brightlab 0x000000010c54cd56 libloading::os::unix::with_dlerror<(),closure> + 134 (mod.rs:38)
7 brightlab 0x000000010c54cf7d _$LT$libloading..os..unix..Library$u20$as$u20$core..ops..Drop$GT$::drop::hd8137c4da21c7d9d + 45 (mod.rs:38)
8 brightlab 0x000000010c4908e1 drop::h9ed5b642e5309eab + 17
9 brightlab 0x000000010c48e409 drop::h2bbca3905b6b58a1 + 9
10 brightlab 0x000000010c490e28 drop::haca7bed875860903 + 72
11 brightlab 0x000000010c48dee1 drop::h1e1054b6f31067c0 + 17
12 brightlab 0x000000010c48f385 drop::h529da6d106e4933f + 149
13 brightlab 0x000000010c4b2313 brightlab::main + 835 (main.rs:37)
14 brightlab 0x000000010c55997b __rust_maybe_catch_panic + 27 (lib.rs:106)
15 brightlab 0x000000010c558ec7 std::rt::lang_start::hefd96b70277e8a4a + 391 (rt.rs:57)
16 brightlab 0x000000010c4b245a main + 42
17 libdyld.dylib 0x00007fff911445c9 start + 1

[nagisa]

Lack of _tlv_atexit in the backtrace seems to suggest that yours is different issue. This comment might help to give some pointers.

[solarretrace]

Hmm, I got the impression from that comment that it would be fixed by default... I'll look into it, thanks.

[Mark-Simulacrum]

Copying code into here so it doesn't get lost; nagisa/rust_libloading#5 is potentially relevant.

test.rs:

#[no_mangle]
pub extern "system" fn test_fn() -> i32 {
    // Removing this line prevents the segfault.
    // I've tried flushing stdout as well but it doesn't change anything
    println!("In library!");
    123456
}

main.c:

#include <dlfcn.h>
#include <stdio.h>

int main() {
    printf("running\n");
    void* handle = dlopen("./libtest.dylib", RTLD_LAZY);
    printf("opened: %p\n", handle);

    int (*test_fn)() = dlsym(handle, "test_fn");
    printf("test_fn: %d\n", test_fn());

    printf("Closing...\n");
    int code = dlclose(handle); // Removing this line prevents the segfault upon exit?
    printf("Closed: %d.\n", code);
}

$ rustc --crate-type=dylib test.rs
$ gcc main.c
$ ./a.out
running
opened: 0x7f8a93c02640
In library!
test_fn: 123456
Closing...
Closed: 0.
Segmentation fault: 11