Windows: Use anonymous pipes in Command#142517

ChrisDenton · 2025-06-14T18:06:59Z

When setting Stdio::pipe on Command we want to create an anonymous pipe that can be used asynchronously (at least on our end). Usually we'd use CreatePipe to open anonymous pipes but unfortunately it opens pipes for synchronous access. The alternative is to use CreateNamedPipeW which does allow asynchronous access but that requires giving a file name to the pipe. So we currently have this awful hack where we attempt to emulate anonymous pipes using CreateNamedPipeW by attempting to create a unique name and looping until we find one that doesn't already exist.

The better option is to use the lower level NtCreateNamedPipeFile (which is used internally by both CreatePipe and CreateNamedPipeW). This function wasn't documented until a few years ago but now that it is it's ok for us to use it.

try-job: msvc
try-job: mingw

rustbot · 2025-06-14T18:07:03Z

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Windows: Use anonymous pipes in Command When setting `Stdio::pipe` on `Command` we want to create an anonymous pipe that can be used asynchronously (at least on our end). Usually we'd use [`CreatePipe`](https://learn.microsoft.com/en-us/windows/win32/api/namedpipeapi/nf-namedpipeapi-createpipe) to open anonymous pipes but unfortunately it opens pipes for synchronous access. The alternative is to use [`CreateNamedPipeW`](https://learn.microsoft.com/en-us/windows/win32/api/namedpipeapi/nf-namedpipeapi-createnamedpipew) which does allow asynchronous access but that requires giving a file name to the pipe. So we currently have this awful hack where we attempt to emulate anonymous pipes using `CreateNamedPipeW` by attempting to create a unique name and looping until we find one that doesn't already exist. The better option is to use the lower level [`NtCreateNamedPipeFile`](https://learn.microsoft.com/en-us/windows/win32/devnotes/nt-create-named-pipe-file) (which is used internally by both `CreatePipe` and `CreateNamedPipeW`). This function wasn't documented until a few years ago but now that it is it's ok for us to use it. try-job: *msvc* try-job: *mingw*

ChrisDenton · 2025-06-14T18:20:36Z

@bors try

bors · 2025-06-14T18:21:46Z

⌛ Trying commit 82f68e0 with merge 4b91445...

Windows: Use anonymous pipes in Command When setting `Stdio::pipe` on `Command` we want to create an anonymous pipe that can be used asynchronously (at least on our end). Usually we'd use [`CreatePipe`](https://learn.microsoft.com/en-us/windows/win32/api/namedpipeapi/nf-namedpipeapi-createpipe) to open anonymous pipes but unfortunately it opens pipes for synchronous access. The alternative is to use [`CreateNamedPipeW`](https://learn.microsoft.com/en-us/windows/win32/api/namedpipeapi/nf-namedpipeapi-createnamedpipew) which does allow asynchronous access but that requires giving a file name to the pipe. So we currently have this awful hack where we attempt to emulate anonymous pipes using `CreateNamedPipeW` by attempting to create a unique name and looping until we find one that doesn't already exist. The better option is to use the lower level [`NtCreateNamedPipeFile`](https://learn.microsoft.com/en-us/windows/win32/devnotes/nt-create-named-pipe-file) (which is used internally by both `CreatePipe` and `CreateNamedPipeW`). This function wasn't documented until a few years ago but now that it is it's ok for us to use it. try-job: *msvc* try-job: *mingw*

bors · 2025-06-14T20:59:28Z

☀️ Try build successful - checks-actions
Build commit: 4b91445 (4b91445f185574b3733649977aa2409da29f288b)

Mark-Simulacrum · 2025-06-15T15:40:03Z

@@ -65,89 +60,108 @@ pub fn anon_pipe(ours_readable: bool, their_handle_inheritable: bool) -> io::Res
    // operations. Instead, we create a "hopefully unique" name and create a
    // named pipe which has overlapped operations enabled.


This comment seems stale?

Oh right, yes. I'll fix that.

Mark-Simulacrum · 2025-06-15T15:41:42Z

+
+            // A negative timeout value is a relative time (rather than an absolute time).
+            // The time is given in 100's of nanoseconds so this is 50 milliseconds.
+            let timeout = (50_i64 * 10000).neg() as u64;


Can you say more about why 50ms? I think this is new -- right?

I would sort of expect that we wouldn't timeout at all...

We were using 0 from CreateNamedPipeW which is a documented as using the default of 50 ms (see nDefaultTimeOut):

A value of zero will result in a default time-out of 50 milliseconds.

The timeout only has an effect when using wait functions like WaitNamedPipeW We don't actually make use of it ourselves but it is possible for users to get at the underlying handle via Child and calling as_handle on one of the fields. We don't make any guarantees here but I think it's worth not changing unless we need to.

I see, agreed that this isn't the PR to change that. Maybe worth adding a note to the comment reflecting where the 50ms came from?

Mark-Simulacrum · 2025-06-15T15:42:35Z

This function wasn't documented until a few years ago but now that it is it's ok for us to use it.

The function is guaranteed to have existed on all supported Windows targets?

ChrisDenton · 2025-06-15T15:53:00Z

This function wasn't documented until a few years ago but now that it is it's ok for us to use it.

The function is guaranteed to have existed on all supported Windows targets?

Yes. It goes back to at least Windows 7 and almost certainly before that.

Mark-Simulacrum · 2025-06-15T15:57:21Z

r=me with nits/comments fixed

ChrisDenton · 2025-06-15T16:54:52Z

@bors r=Mark-Simulacrum

bors · 2025-06-15T16:54:56Z

📌 Commit 5dc235f has been approved by Mark-Simulacrum

It is now in the queue for this repository.

ChrisDenton · 2025-06-15T16:55:19Z

Oh, wait I didn't push the second comment.

@bors r-

ChrisDenton · 2025-06-15T16:58:05Z

@bors r=Mark-Simulacrum

bors · 2025-06-15T16:58:08Z

📌 Commit 0e1db54 has been approved by Mark-Simulacrum

It is now in the queue for this repository.

…lacrum Windows: Use anonymous pipes in Command When setting `Stdio::pipe` on `Command` we want to create an anonymous pipe that can be used asynchronously (at least on our end). Usually we'd use [`CreatePipe`](https://learn.microsoft.com/en-us/windows/win32/api/namedpipeapi/nf-namedpipeapi-createpipe) to open anonymous pipes but unfortunately it opens pipes for synchronous access. The alternative is to use [`CreateNamedPipeW`](https://learn.microsoft.com/en-us/windows/win32/api/namedpipeapi/nf-namedpipeapi-createnamedpipew) which does allow asynchronous access but that requires giving a file name to the pipe. So we currently have this awful hack where we attempt to emulate anonymous pipes using `CreateNamedPipeW` by attempting to create a unique name and looping until we find one that doesn't already exist. The better option is to use the lower level [`NtCreateNamedPipeFile`](https://learn.microsoft.com/en-us/windows/win32/devnotes/nt-create-named-pipe-file) (which is used internally by both `CreatePipe` and `CreateNamedPipeW`). This function wasn't documented until a few years ago but now that it is it's ok for us to use it. try-job: *msvc* try-job: *mingw*

Rollup of 11 pull requests Successful merges: - #140809 (Reduce special casing for the panic runtime) - #141608 (Add support for repetition to `proc_macro::quote`) - #141864 (Handle win32 separator for cygwin paths) - #142216 (Miscellaneous RefCell cleanups) - #142517 (Windows: Use anonymous pipes in Command) - #142570 (Reject union default field values) - #142584 (Handle same-crate macro for borrowck semicolon suggestion) - #142585 (Update books) - #142586 (Fold unnecessary `visit_struct_field_def` in AstValidator) - #142595 (Revert overeager warning for misuse of `--print native-static-libs`) - #142598 (Set elf e_flags on ppc64 targets according to abi) r? `@ghost` `@rustbot` modify labels: rollup

…lacrum Windows: Use anonymous pipes in Command When setting `Stdio::pipe` on `Command` we want to create an anonymous pipe that can be used asynchronously (at least on our end). Usually we'd use [`CreatePipe`](https://learn.microsoft.com/en-us/windows/win32/api/namedpipeapi/nf-namedpipeapi-createpipe) to open anonymous pipes but unfortunately it opens pipes for synchronous access. The alternative is to use [`CreateNamedPipeW`](https://learn.microsoft.com/en-us/windows/win32/api/namedpipeapi/nf-namedpipeapi-createnamedpipew) which does allow asynchronous access but that requires giving a file name to the pipe. So we currently have this awful hack where we attempt to emulate anonymous pipes using `CreateNamedPipeW` by attempting to create a unique name and looping until we find one that doesn't already exist. The better option is to use the lower level [`NtCreateNamedPipeFile`](https://learn.microsoft.com/en-us/windows/win32/devnotes/nt-create-named-pipe-file) (which is used internally by both `CreatePipe` and `CreateNamedPipeW`). This function wasn't documented until a few years ago but now that it is it's ok for us to use it. try-job: *msvc* try-job: *mingw*

Rollup of 10 pull requests Successful merges: - #138538 (Make performance description of String::{insert,insert_str,remove} more precise) - #141946 (std: refactor explanation of `NonNull`) - #142216 (Miscellaneous RefCell cleanups) - #142371 (avoid `&mut P<T>` in `visit_expr` etc methods) - #142377 (Try unremapping compiler sources) - #142517 (Windows: Use anonymous pipes in Command) - #142542 (Manually invalidate caches in SimplifyCfg.) - #142563 (Refine run-make test ignores due to unpredictable `i686-pc-windows-gnu` unwind mechanism) - #142570 (Reject union default field values) - #142584 (Handle same-crate macro for borrowck semicolon suggestion) r? `@ghost` `@rustbot` modify labels: rollup

Rollup of 10 pull requests Successful merges: - #138538 (Make performance description of String::{insert,insert_str,remove} more precise) - #141946 (std: refactor explanation of `NonNull`) - #142216 (Miscellaneous RefCell cleanups) - #142371 (avoid `&mut P<T>` in `visit_expr` etc methods) - #142377 (Try unremapping compiler sources) - #142517 (Windows: Use anonymous pipes in Command) - #142542 (Manually invalidate caches in SimplifyCfg.) - #142563 (Refine run-make test ignores due to unpredictable `i686-pc-windows-gnu` unwind mechanism) - #142570 (Reject union default field values) - #142584 (Handle same-crate macro for borrowck semicolon suggestion) r? `@ghost` `@rustbot` modify labels: rollup  [Create a similar rollup](https://bors.rust-lang.org/queue/rust?prs=138538,141946,142216,142371,142377,142517,142542,142563,142570,142584)  try-job: dist-aarch64-apple

Rollup of 14 pull requests Successful merges: - #141574 (impl `Default` for `array::IntoIter`) - #141608 (Add support for repetition to `proc_macro::quote`) - #142100 (rustdoc: make srcIndex no longer a global variable) - #142371 (avoid `&mut P<T>` in `visit_expr` etc methods) - #142517 (Windows: Use anonymous pipes in Command) - #142520 (alloc: less static mut + some cleanup) - #142588 (Generic ctx imprv) - #142605 (Don't unwrap in enzyme builds in case of missing llvm-config) - #142608 (Refresh module-level docs for `rustc_target::spec`) - #142618 (Lint about `console` calls in rustdoc JS) - #142620 (Remove a panicking branch in `BorrowedCursor::advance`) - #142631 (Dont suggest remove semi inside macro expansion for redundant semi lint) - #142632 (Update cargo) - #142635 (Temporarily add back -Zwasm-c-abi=spec) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of #142517 - ChrisDenton:anon-pipe, r=Mark-Simulacrum Windows: Use anonymous pipes in Command When setting `Stdio::pipe` on `Command` we want to create an anonymous pipe that can be used asynchronously (at least on our end). Usually we'd use [`CreatePipe`](https://learn.microsoft.com/en-us/windows/win32/api/namedpipeapi/nf-namedpipeapi-createpipe) to open anonymous pipes but unfortunately it opens pipes for synchronous access. The alternative is to use [`CreateNamedPipeW`](https://learn.microsoft.com/en-us/windows/win32/api/namedpipeapi/nf-namedpipeapi-createnamedpipew) which does allow asynchronous access but that requires giving a file name to the pipe. So we currently have this awful hack where we attempt to emulate anonymous pipes using `CreateNamedPipeW` by attempting to create a unique name and looping until we find one that doesn't already exist. The better option is to use the lower level [`NtCreateNamedPipeFile`](https://learn.microsoft.com/en-us/windows/win32/devnotes/nt-create-named-pipe-file) (which is used internally by both `CreatePipe` and `CreateNamedPipeW`). This function wasn't documented until a few years ago but now that it is it's ok for us to use it. try-job: *msvc* try-job: *mingw*

Rollup of 14 pull requests Successful merges: - rust-lang/rust#141574 (impl `Default` for `array::IntoIter`) - rust-lang/rust#141608 (Add support for repetition to `proc_macro::quote`) - rust-lang/rust#142100 (rustdoc: make srcIndex no longer a global variable) - rust-lang/rust#142371 (avoid `&mut P<T>` in `visit_expr` etc methods) - rust-lang/rust#142517 (Windows: Use anonymous pipes in Command) - rust-lang/rust#142520 (alloc: less static mut + some cleanup) - rust-lang/rust#142588 (Generic ctx imprv) - rust-lang/rust#142605 (Don't unwrap in enzyme builds in case of missing llvm-config) - rust-lang/rust#142608 (Refresh module-level docs for `rustc_target::spec`) - rust-lang/rust#142618 (Lint about `console` calls in rustdoc JS) - rust-lang/rust#142620 (Remove a panicking branch in `BorrowedCursor::advance`) - rust-lang/rust#142631 (Dont suggest remove semi inside macro expansion for redundant semi lint) - rust-lang/rust#142632 (Update cargo) - rust-lang/rust#142635 (Temporarily add back -Zwasm-c-abi=spec) r? `@ghost` `@rustbot` modify labels: rollup

…lacrum Windows: Use anonymous pipes in Command When setting `Stdio::pipe` on `Command` we want to create an anonymous pipe that can be used asynchronously (at least on our end). Usually we'd use [`CreatePipe`](https://learn.microsoft.com/en-us/windows/win32/api/namedpipeapi/nf-namedpipeapi-createpipe) to open anonymous pipes but unfortunately it opens pipes for synchronous access. The alternative is to use [`CreateNamedPipeW`](https://learn.microsoft.com/en-us/windows/win32/api/namedpipeapi/nf-namedpipeapi-createnamedpipew) which does allow asynchronous access but that requires giving a file name to the pipe. So we currently have this awful hack where we attempt to emulate anonymous pipes using `CreateNamedPipeW` by attempting to create a unique name and looping until we find one that doesn't already exist. The better option is to use the lower level [`NtCreateNamedPipeFile`](https://learn.microsoft.com/en-us/windows/win32/devnotes/nt-create-named-pipe-file) (which is used internally by both `CreatePipe` and `CreateNamedPipeW`). This function wasn't documented until a few years ago but now that it is it's ok for us to use it. try-job: *msvc* try-job: *mingw*

lhecker · 2026-04-10T17:43:05Z

+        // Open a handle to the pipe filesystem (`\??\PIPE\`).
+        // This will be used when creating a new annon pipe.
+        let pipe_fs = {
+            let path = c::UNICODE_STRING::from_ref(utf16!(r"\??\PIPE\"));


I just came across this PR, as I was happy to see Rust adopt anonymous overlapped pipes.

This line should probably? technically use \Device\NamedPipe\. Even the Win32 code uses the direct NT path instead of the DOS symlink. (I work for MSFT and have source access. The internal code also caches the driver handle for the duration of the process, but I do not know how important that is.)

Is there a significant improvement to using the device directly instead of the symlink? We tend to prefer using the "high level" win32 stuff to the extent possible due to (at least perceived) better compatibility (the Windows DLLs are of course shipped with the OS so can change with it), The exception is when there's a practical advantage to doing otherwise. Caching the handle (which seems like a good idea!) sounds like it would mitigate any performance difference so I'm unsure what the advantage would be,

There's definitely some overhead to it but it's not too much:

----------------------------------------------------- Benchmark Time CPU Iterations ----------------------------------------------------- Cached 5306 ns 5312 ns 100000 Uncached 7316 ns 7324 ns 89600

Personally, I would cache it but primarily only because the internal Windows source code for CreatePipe caches it in a static variable and I assume that AV scanners (for instance) may be basing their fairly restrictive decisions on it. At Windows Terminal (the main project I'm working on) we've definitely noticed time and time again that AV scanners cause major overheads when it comes to overlapped pipe creation and IO.

In any case, for the same reason I would probably tend to use \Device\NamedPipe\. Not because I can tell you whether it's better or worse (I expect it to be identical), but simply because the CreatePipe function in kernelbase.dll uses it and has been using it since the creation of the NT kernel. In that vein, using the symlink is more unusual IMO. I'd consider it a minor nit perhaps, but worth considering.

If it's worth anything, Windows Terminal uses it - I also believe we were the first to publish this method? https://github.com/microsoft/terminal/blob/84ae7adec6b3975314d8ca73d8f0bf2128ae59e2/src/types/utils.cpp#L785-L885

rustbot assigned Mark-Simulacrum Jun 14, 2025

rustbot added O-windows Operating system: Windows S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Jun 14, 2025

This comment was marked as outdated.

Sign in to view

ChrisDenton force-pushed the anon-pipe branch from 529881a to 82f68e0 Compare June 14, 2025 18:15

This comment was marked as outdated.

Sign in to view

Mark-Simulacrum reviewed Jun 15, 2025

View reviewed changes

ChrisDenton force-pushed the anon-pipe branch from 82f68e0 to 5dc235f Compare June 15, 2025 16:19

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 15, 2025

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jun 15, 2025

Windows: Use anonymous pipes in Command

0e1db54

ChrisDenton force-pushed the anon-pipe branch from 5dc235f to 0e1db54 Compare June 15, 2025 16:57

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 15, 2025

jhpratt mentioned this pull request Jun 17, 2025

Rollup of 11 pull requests #142604

Closed

workingjubilee mentioned this pull request Jun 17, 2025

Rollup of 10 pull requests #142614

Closed

jhpratt mentioned this pull request Jun 17, 2025

Rollup of 14 pull requests #142644

Merged

bors merged commit b5fcc90 into rust-lang:master Jun 18, 2025
10 checks passed

rustbot added this to the 1.89.0 milestone Jun 18, 2025

seritools mentioned this pull request Jan 1, 2026

Rust9x updates for future Rust versions rust9x/rust#47

Closed

2 tasks

ChrisDenton deleted the anon-pipe branch January 2, 2026 00:05

lhecker reviewed Apr 10, 2026

View reviewed changes

ChrisDenton mentioned this pull request Apr 13, 2026

Windows: Cache the pipe filesystem handle #155250

Open

		@@ -65,89 +60,108 @@ pub fn anon_pipe(ours_readable: bool, their_handle_inheritable: bool) -> io::Res
		// operations. Instead, we create a "hopefully unique" name and create a
		// named pipe which has overlapped operations enabled.

Uh oh!

Conversation

ChrisDenton commented Jun 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Jun 14, 2025

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

ChrisDenton commented Jun 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bors commented Jun 14, 2025

Uh oh!

bors commented Jun 14, 2025

Uh oh!

Mark-Simulacrum Jun 15, 2025

Choose a reason for hiding this comment

Uh oh!

ChrisDenton Jun 15, 2025

Choose a reason for hiding this comment

Uh oh!

Mark-Simulacrum Jun 15, 2025

Choose a reason for hiding this comment

Uh oh!

ChrisDenton Jun 15, 2025

Choose a reason for hiding this comment

Uh oh!

Mark-Simulacrum Jun 15, 2025

Choose a reason for hiding this comment

Uh oh!

Mark-Simulacrum commented Jun 15, 2025

Uh oh!

ChrisDenton commented Jun 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Mark-Simulacrum commented Jun 15, 2025

Uh oh!

ChrisDenton commented Jun 15, 2025

Uh oh!

bors commented Jun 15, 2025

Uh oh!

ChrisDenton commented Jun 15, 2025

Uh oh!

ChrisDenton commented Jun 15, 2025

Uh oh!

bors commented Jun 15, 2025

Uh oh!

Uh oh!

lhecker Apr 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ChrisDenton Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

lhecker Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ChrisDenton commented Jun 14, 2025 •

edited

Loading

ChrisDenton commented Jun 14, 2025 •

edited

Loading

ChrisDenton commented Jun 15, 2025 •

edited

Loading

lhecker Apr 10, 2026 •

edited

Loading