std::process::Command: piped stderr hangs in Windows

[dherman]

It looks like the use of overlapped I/O on stdio ports may cause some problems in Windows. When I try to shell out to npm install using Stdio::piped() on the stderr port, the command runs successfully but hangs at the end.

@alexcrichton pointed me to this commit as the likely culprit: 7c3038f

Here's the smallest test case I could figure out how to create: https://github.com/dherman/hang-command

I wanted to try to come up with a minimal Node script to shell out to, as opposed to npm install. Unfortunately, I'm not good enough at debugging npm and Node to figure out exactly what they're doing that causes the hang.

[alexcrichton]

Ok I've tested this and it's a stable-to-stable regression that happened between 1.8 and 1.9. I believe the 1.9 release is indeed the first one with #31618, the likely change which is a culprit. Tagging appropriately as such.

[rprichard]

It looks like Rust is passing overlapped-mode pipe handles to the child process. The pipe::anon_pipe function returns two overlapped HANDLES. I'm not sure the overlapped mode can be changed after the fact -- maybe ReOpenFile would work -- but I don't see it used anywhere, and I'd wonder about its use on the server handle.

If the child inherits overlapped-mode handles, it would need to do async I/O to stdin/stdout/stderr. By convention, it does sync I/O using lpOverlapped==NULL, but that has undefined behavior if the handles are overlapped pipes.

https://blogs.msdn.microsoft.com/oldnewthing/20120411-00/?p=7883
https://blogs.msdn.microsoft.com/oldnewthing/20121012-00/?p=6343

FWIW: the server and client ends of a named pipe aren't quite symmetric, but I'm guessing it doesn't matter whether the child process receives a server pipe or a client pipe? The interesting thing is that writing to a broken pipe returns ERROR_NO_DATA, but reading from one returns ERROR_BROKEN_PIPE. Apparently this is true for both ends. (I apparently wrote an extensive test program for this back in June'16.) I'd probably pass the client end to the child process, because it doesn't make sense that a program can call DisconnectNamedPipe on its stdio handles. (DisconnectNamedPipe doesn't work on client handles.)

I made a Rust + C++ test case demonstrating how passing overlapped handles to the child breaks:
https://gist.github.com/rprichard/37bfd1dd6a2ef91ba7af49efbe0461ed

[alexcrichton]

Fascinating! Thanks for the links @rprichard! That perfectly explains I think what's going on here.

I think I can see the fix here, so I'll work on that pronto.

[rprichard]

Yeah, I think the fix is easy -- use non-overlapped handles for the child but overlapped handles for the parent.