recognize $PATHEXT on windows for cargo subcommands

[insomnimus]

Problem

Currently on Windows, external cargo subcommands must have the .exe extension for cargo to recognize them.

However, .exe files are not the only executable extensions as you already know.

Proposed Solution

What many don't know is, there's a standardized environment variable Windows uses to determine if files can be executed from $PATH, which is also pre-populated by Windows: $PATHEXT.

The contents of $PATHEXT might look like this:

.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JSE;.WSF;.WSH;.MSC;.CPL

The user is free to extend this value by adding custom extensions, and the "interpreter" is likewise configurable through some built-in cmd.exe commands or the Windows registry.

On Linux or MacOSX file extensions play no role so the user can write their cargo extensions in a bash script with a shebang and set the executable bit for cargo to recognize it (assuming the file is named cargo-* and available under $PATH).

Similar can be accomplished by respecting the value of $PATHEXT on Windows, or, not specifying *.exe since programs with an extension contained in $PATHEXT don't require the extension to be specified.

Notes

No response

[weihanglo]

Interesting! I didn't know this before, so I searched it in rust-lang/rust and found some existing issues:

• Find the program using PATHEXT rust#37381
• Command::spawn has weird rules for finding binaries on Windows rust#37519

As far as I understand it, the biggest problem is that at this time std::process::Command tries to provide closely how each platform creates a process. On windows, it uses CreateProcess¹ and it doesn't recognize PATHEXT at all. PATHEXT seems like an exclusive feature of cmd.exe.

If we want to utilize PATHEXT. Cargo needs to do something like Command::new("cmd.exe").arg("/c").arg("foo").spawn(). I am not familiar with Windows but there might be some pitfalls I didn't aware of. Though I guess it will affect some existing program if cargo starts to spawn command via cmd.exe.

As a reference, here is the code finding the external commands in cargo:

cargo/src/bin/cargo/main.rs

Lines 158 to 164 in 0a3f2b4

	fn find_external_subcommand(config: &Config, cmd: &str) -> Option<PathBuf> {
	let command_exe = format!("cargo-{}{}", cmd, env::consts::EXE_SUFFIX);
	search_directories(config)
	.iter()
	.map(|dir| dir.join(&command_exe))
	.find(|file| is_executable(file))
	}

Footnotes

1. https://msdn.microsoft.com/en-us/library/windows/desktop/ms682425(v=vs.85).aspx ↩

[insomnimus]

Thanks for the detailed response!

It seems you're correct that $PATHEXT is a cmd.exe variable (although it seems like powershell uses it as well).

Obviously integrating this to the stdlib is out of scope and frankly, doesn't make a lot of sense.

What if we had a cargo configuration key for subcommand lookup extensions?

I can see this used only on Windows but it might be desirable.

Let me know what you think.

[insomnimus]

Oh, here is how one would configure custom interpreters for file extensions on the command line (cmd.exe, these are built-ins so powershell won't work):

It's a two-step process.

I'll demonstrate a trick I use to run shell scripts with wsl.

Below snippets (assoc and ftype) run on an elevated cmd.exe prompt.

First we associate an extension with a tag of sorts:

C:\> assoc .sh=shell.script

With the "tag" configured, we add a handler for it:

C:\> ftype shell.script=C:\system32\wsl.exe "%1" "%*"

Actually, it's 3 steps, at least on the command line.
We have to append .sh to the $PATHEXT env variable.

PS C:\> $env:PATHEXT += ";.sh"

Now, on powershell, shell scripts under $PATH can be executed without specifying the .sh extension.

Makes one appreciate shebangs a lot, doesn't it :D.

[insomnimus]

I've done some experimentation.

It looks like process::Command does not make use of the registry keys for file associations, so non-builtin extensions (like .py) can't be executed through the standard process APIs.

For these to work, I assume we would have to inspect the registry, which can be done but probably not worth it.

Still though, batch files (.bat, .cmd...) work when invoked with process::Command so at least we could support these?

[ChrisDenton]

The proper way to execute an non-exe file is to use ShellExecuteW. But this is higher level and may require COM initialization, which is why the standard library has so far not used it.

The way it searches file associations for the executable to use could also be implemented manually (see docs for registry keys, etc) but that's complex enough that it should almost certainly be tested as a third party crate before deciding whether to incorporate it into standard library.

So this could be done but unfortunately the standard library isn't going to be too much help at this time.

[insomnimus]

I agree.

How about at least supporting extensions std::process::Command already supports?

I have tested with .bat and .cmd, they both work the same way as an exe file.

I would be interested in implementing it myself, if that's fine.

[ChrisDenton]

This is undocumented behaviour that just so happens to work. The MSDN docs say you should instead do this:

To run a batch file, you must start the command interpreter; set lpApplicationName to cmd.exe and set lpCommandLine to the following arguments: /c plus the name of the batch file.

The other issue is passing additional arguments. Argument parsing works differently when running cmd.exe. Though the currently unstable raw_arg might help here.

[joshtriplett]

@insomnimus We'd be up for supporting what std::process::Command already supports, and extending cargo list to handle a few more extensions, as long as we don't have to manually implement the mechanism for invoking a bat/cmd file via the interpreter. (Such a mechanism should live in std::process::Command.)

[ChrisDenton]

I've created an issue on the rust repo where I've tried to outline what's needed to properly support this (see rust-lang/rust#94743). For .bat and .cmd scripts, the main work would be creating tests to make sure arguments passed to the batch file are interpreted correctly.

[weihanglo]

Regarding the fresh CVE-2024-24576 and the following discussion in rust-lang/rust#123728, I wonder if we should close this as “not supported”. See also the documentation in std::process about Windows argument splitting.

[epage]

I second this. rust-lang/rust#94743 was rejected and others expressed dependence on what the standard library chooses to do.