Wrong calling convention on using ffi of extern mod

[klutzy]

Here's an example on win32.

a.rs:

#[crate_type = "lib"];

pub use std::libc::*;
pub use std::libc::types::os::arch::extra::*;

extern "stdcall" {
    fn MessageBoxW(hWnd: HANDLE, lpText: LPCWSTR, lpCaption: LPCWSTR, uType: c_uint) -> c_int;
}

Build a.rs as shared library.

b.rs:

extern mod a;
use std::ptr;

#[fixed_stack_segment]
fn main() {
    unsafe { a::MessageBoxW(ptr::mut_null(), ptr::null(), ptr::null(), 0 as a::c_uint); }
}

Building b.rs I got linking failure:

note: b.o:fake:(.text+0x86): undefined reference to `MessageBoxW'

LLVM IR has some interesting point:

; Function Attrs: noinline uwtable
define void @_ZN4main16_bd1837541cdc5544v0.0E({ i32, %tydesc*, i8*, i8*, i8 }*) #4 {
  ...
  %14 = call x86_stdcallcc i32 @MessageBoxW(%"enum.std::libc::types::common::c95::c_void[#1]"* %10, i16* %11, i16* %12, i32 %13)
  ...
}
...
declare i32 @MessageBoxW(%"enum.std::libc::types::common::c95::c_void[#1]"*, i16*, i16*, i32)

so rustc generates correct call, but wrong declare (no callconv).

Frankly I don't even know if it's ok to use foreign fn from extern module. I'm also confused that there is no need to add pub at MessageBoxW declaration.

[klutzy]

(The example worked on 0.7.)

[nikomatsakis]

Interesting. cc me.

[klutzy]

I tried to make a workaround by wrapping MessageBoxW in this example. However, it is still problematic for certain cases.

a.rs:

#[crate_type = "lib"];

pub use std::ptr;
pub use std::libc::*;
pub use std::libc::types::os::arch::extra::*;

extern "stdcall" {
    fn MessageBoxW(hWnd: HANDLE, lpText: LPCWSTR, lpCaption: LPCWSTR, uType: c_uint) -> c_int;
}

pub trait Dummy {}

pub trait Window {
    fn message_box(&self);
}

impl<D: Dummy> Window for D {
    fn message_box(&self) {
        #[fixed_stack_segment];
        unsafe { MessageBoxW(ptr::mut_null(), ptr::null(), ptr::null(), 0 as c_uint); }
    }
}

b.rs:

extern mod a;
use a::*;

struct D;
impl Dummy for D;

fn main() {
    let x = D;
    x.message_box();
}

which occurs same callconv issue.

[klutzy]

It occurs on generic functions, not only on generic impls:

a.rs:

#[crate_type = "lib"];

use std::ptr;

pub use std::libc::*;
pub use std::libc::types::os::arch::extra::*;

extern "stdcall" {
    fn MessageBoxW(hWnd: HANDLE, lpText: LPCWSTR, lpCaption: LPCWSTR, uType: c_uint) -> c_int;
}

pub fn g<T>() {
    #[fixed_stack_segment];
    unsafe { MessageBoxW(ptr::mut_null(), ptr::null(), ptr::null(), 0 as c_uint); }
}

b.rs:

extern mod a;

fn main() {
    a::g::<int>();
}

This means std::unstable is totally unusable on win32 because it has pub fn check_for_errors_in<T>(f: &fn()->T) -> Result<T, ~str>.
This blocks #8540. cc @cmr

[klutzy]

From #8203 bors log, it is broken somewhere between 0.7 and 2246d56.
@nikomatsakis is it possible #8535 related to this?

[thestinger]

You could narrow it down with git bisect, the window between the commits is quite large.

[klutzy]

#9196 fixed all three examples. closing!