std::process::Command: piped stderr hangs in Windows #38811
Comments
|
Ok I've tested this and it's a stable-to-stable regression that happened between 1.8 and 1.9. I believe the 1.9 release is indeed the first one with #31618, the likely change which is a culprit. Tagging appropriately as such. |
alexcrichton
added
regression-from-stable-to-stable
|
It looks like Rust is passing overlapped-mode pipe handles to the child process. The If the child inherits overlapped-mode handles, it would need to do async I/O to stdin/stdout/stderr. By convention, it does sync I/O using lpOverlapped==NULL, but that has undefined behavior if the handles are overlapped pipes. https://blogs.msdn.microsoft.com/oldnewthing/20120411-00/?p=7883 FWIW: the server and client ends of a named pipe aren't quite symmetric, but I'm guessing it doesn't matter whether the child process receives a server pipe or a client pipe? The interesting thing is that writing to a broken pipe returns I made a Rust + C++ test case demonstrating how passing overlapped handles to the child breaks: |
|
Fascinating! Thanks for the links @rprichard! That perfectly explains I think what's going on here. I think I can see the fix here, so I'll work on that pronto. |
|
Yeah, I think the fix is easy -- use non-overlapped handles for the child but overlapped handles for the parent. |
This commit fixes a mistake introduced in rust-lang#31618 where overlapped handles were leaked to child processes on Windows. On Windows once a handle is in overlapped mode it should always have I/O executed with an instance of `OVERLAPPED`. Most child processes, however, are not prepared to have their stdio handles in overlapped mode as they don't use `OVERLAPPED` on reads/writes to the handle. Now we haven't had any odd behavior in Rust up to this point, and the original bug was introduced almost a year ago. I believe this is because it turns out that if you *don't* pass an `OVERLAPPED` then the system will [supply one for you][link]. In this case everything will go awry if you concurrently operate on the handle. In Rust, however, the stdio handles are always locked, and there's no way to not use them unlocked in libstd. Due to that change we've always had synchronized access to these handles, which means that Rust programs typically "just work". Conversely, though, this commit fixes the test case included, which exhibits behavior that other programs Rust spawns may attempt to execute. Namely, the stdio handles may be concurrently used and having them in overlapped mode wreaks havoc. [link]: https://blogs.msdn.microsoft.com/oldnewthing/20121012-00/?p=6343 Closes rust-lang#38811
This commit fixes a mistake introduced in rust-lang#31618 where overlapped handles were leaked to child processes on Windows. On Windows once a handle is in overlapped mode it should always have I/O executed with an instance of `OVERLAPPED`. Most child processes, however, are not prepared to have their stdio handles in overlapped mode as they don't use `OVERLAPPED` on reads/writes to the handle. Now we haven't had any odd behavior in Rust up to this point, and the original bug was introduced almost a year ago. I believe this is because it turns out that if you *don't* pass an `OVERLAPPED` then the system will [supply one for you][link]. In this case everything will go awry if you concurrently operate on the handle. In Rust, however, the stdio handles are always locked, and there's no way to not use them unlocked in libstd. Due to that change we've always had synchronized access to these handles, which means that Rust programs typically "just work". Conversely, though, this commit fixes the test case included, which exhibits behavior that other programs Rust spawns may attempt to execute. Namely, the stdio handles may be concurrently used and having them in overlapped mode wreaks havoc. [link]: https://blogs.msdn.microsoft.com/oldnewthing/20121012-00/?p=6343 Closes rust-lang#38811
std: Don't pass overlapped handles to processes This commit fixes a mistake introduced in #31618 where overlapped handles were leaked to child processes on Windows. On Windows once a handle is in overlapped mode it should always have I/O executed with an instance of `OVERLAPPED`. Most child processes, however, are not prepared to have their stdio handles in overlapped mode as they don't use `OVERLAPPED` on reads/writes to the handle. Now we haven't had any odd behavior in Rust up to this point, and the original bug was introduced almost a year ago. I believe this is because it turns out that if you *don't* pass an `OVERLAPPED` then the system will [supply one for you][link]. In this case everything will go awry if you concurrently operate on the handle. In Rust, however, the stdio handles are always locked, and there's no way to not use them unlocked in libstd. Due to that change we've always had synchronized access to these handles, which means that Rust programs typically "just work". Conversely, though, this commit fixes the test case included, which exhibits behavior that other programs Rust spawns may attempt to execute. Namely, the stdio handles may be concurrently used and having them in overlapped mode wreaks havoc. [link]: https://blogs.msdn.microsoft.com/oldnewthing/20121012-00/?p=6343 Closes #38811
… in Windows, eliminate the workaround hack for the rust issue that's been fixed.
It looks like the use of overlapped I/O on stdio ports may cause some problems in Windows. When I try to shell out to
npm installusingStdio::piped()on the stderr port, the command runs successfully but hangs at the end.@alexcrichton pointed me to this commit as the likely culprit: 7c3038f
Here's the smallest test case I could figure out how to create: https://github.com/dherman/hang-command
I wanted to try to come up with a minimal Node script to shell out to, as opposed to
npm install. Unfortunately, I'm not good enough at debugging npm and Node to figure out exactly what they're doing that causes the hang.The text was updated successfully, but these errors were encountered: