Announcement (and discussion) of changes to come...

[dorssel]

History

When usbipd-win was first developed, WSL was not part of the project. Microsoft picked up on the project and added some WSL-specific stuff. This lead to the usbipd wsl ... convenience commands.

usbipd wsl attach is essentially a usbipd bind (but only the first time, which requires admin rights) followed by a remote WSL usbip attach (client side, not requiring admin on Windows).

usbipd wsl detach is essentially a "disconnect" of the TCP connection. It doesn't actually do a detach (which would be graceful) but rather a "yank the device out of the virtual USB port". In fact, WSL is not even called, it does this entirely host-side, and it requires either admin rights or being the interactive user on Windows.

usbipd wsl list was intended to augment the regular usbipd list to display one additional detail: to which distro is your device attached.

Problems in the past

Originally, the WSL stuff was intended to be per-distro. But that never was really the case. When you attach a device to WSL, you attach it to the one-and-only WSL kernel, and the device becomes available in all distros. The usbipd wsl attach --distribution option is just there to choose a distro that has the client tools installed. Once attached, all distros are equal.

usbipd wsl list tried to display to which WSL distro the device was attached. Given the previous paragraph, it is now obvious that this never worked. It tried to match the client IP address to one specific distro, but all distros share the same network, and therefore have the same IP address. In fact, it is impossible to detect which distro was used to execute the attach, and it also doesn't matter.

Previous changes

Recently (version 3.x.y, I cannot remember) I removed the client details from usbipd wsl list, as they were bogus anyway. It used to just always display the first running distro. Currently, usbipd wsl list simply lists "Attached - WSL" or "Attached - non-WSL". Otherwise, there is no difference with the regular usbipd list command.

New problems

WSL keeps changing (for the better...) and new problems arise. For example: WSL creates a network switch with "WSL" in the name. usbipd uses this to determine the host address to be used by the WSL client tools. Now it turns out that Windows Subsystem for Android (WSA) also creates such a switch with a name like "WSL Core". People that are using both WSL and WSA are experiencing problems because (depending on chance) usbipd sometimes finds the WSA switch first, which is of course unreachable from WSL. Furthermore, recent WSL versions (2.0.x) also create a second switch "WSL Firewall". And it depends on your settings in .wslconfig which of the two is actually used.

On top of that, there is now mirrored mode (which is great), but that doesn't use the WSL switch(es) (which still exist), but instead requires to use a localhost address. There are actually two options to select mirrored mode: in the early [experimental] section, and in later in the [wsl2] section (both are honored by recent WSL). But you can also have these options in .wslconfig without running a new WSL versions. In that case they do nothing and the regular WSL switch needs to be used. All-in-all, somewhat of a mess to parse and do right 100%. There is also a command on the Linux side (wslinfo --networking-mode), but the current code of usbipd needs to know before we run WSL commands. usbipd wsl list currently determines if a device is WSL-attached by checking the client IP without running any remote WSL commands. This remote command would slow listing down considerably. Finally, a lot of people use ssh reverse tunnels to bind there device to remote machines. Those appear to be coming from localhost as well (that is how tunnels work), and now cannot be told apart from WSL mirror-mode attaches.

Decisions with rationale

The upcoming version 4.0.0 of usbipd will have some breaking changes:

• usbipd wsl list command will be removed entirely. Currently the only difference with usbipd list is that it will show whether a device is attached to either WSL or to some non-WSL client. With mirrored mode now available, this detection is no longer reliable. Simply removing that information from usbipd wsl list would make the two commands identical. I have decided not to keep both (for "backward compatibility") because: a) the output of usbipd wsl list would change anyway, breaking backward compatibility, and b) I don't like redundant clutter.
• usbipd state (and the PowerShell command Get-UsbipdDevice) will no longer have the field IsWslAttached, as that information is no longer available.
• usbipd wsl detach will become usbipd detach. It works (and already worked in the past) just as well for devices attached to other clients. I.e., it is really not WSL-specific at all. I did consider to give it a different name, as "detach" suggests graceful removal whereas to the client it is what is known as a "suprise removal" in USB-land. But terms like "disconnect", "yank", "unplug" all do not quite cover what it does or would just add to the confusion. We use bind and unbind (because that is how the Linux tools call them) for "sharing" (which is more a Windows term, just like sharing a printer). Likewise we use attach and detach. Even though detach is slightly different (as in: we do not actually call detach from the client, but disconnect from the server), to a user they are each others complement. Therefore, I stuck with usbipd detach.
• usbipd wsl attach will become usbipd attach --wsl [optional-distro-name]. It seems silly to have a wsl subcommand that only has a single command attach (remember that detach and list will be gone from usbipd wsl). Furthermore, it makes sense to provide the attach and detach available at the same command level; since detach is generic now, attach will be at the top level as well. Finally, in the future maybe (just maybe) --wsa and --hyper-v options might become available. You must supply the --wsl (or -w) option, to make clear that usbipd attach (for now) only works for WSL. It is not possible to attach host-side to generic clients, as we have no way of executing a generic remote attach.

[dorssel]

@benmcmorran
Please have a look at this. This will have implications for the documentation.

[TaalExact]

Hi Frans, hier Olav van het Stedelijk Gym. Sorry dat ik je via deze weg benader. Als je wilt, kun je me dan even mailen op [email protected] ?

[dorssel]

@andrewleech @dushanabe
Please have a look this. This will affect your projects.

[dushanabe]

@dorssel Thank you for the heads up, I have now updated the vscode extension to accomodate the changes for version 4.0.0. I've also the extension to be version 3.2.0 compatible in this release with a warning on startup.

[dorssel]

Here is one more ramble about a common issue that will be resolved. I've had a few dozen issues reported by people that run as a standard user, and therefore require admin rights from a different account (instead of elevated rights). When they use usbipd wsl attach for the first time, they are informed that they need admin rights (because of the bind required the first time). Then they run usbipd wsl attach as their (different) admin user, which then (of course) complains that the WSL distro cannot be found, as those are per user. They really need to run usbipd bind first as admin, and then usbipd wsl attach as normal user (which now no longer requires admin as the bind already happened). All clear to us, but not to the average starter.

With the upcoming changes you must first usbipd bind and then usbipd attach --wsl. This makes it obvious when admin rights are required.

Furthermore, somewhere in version 2.x the bind --force option was introduced. That was never carried over to wsl attach. Those who wanted or needed --force had to bind separately anyway.

The changes will make all of this much easier to explain and document:

1. You need (as an admin) to bind your device only once. Either with or without --force. This survives reboots and upgrades.
2. From that moment on you can attach --wsl at will, as a standard user.

[benmcmorran]

Thanks for looping me in. These changes all make sense to me and seem well-reasoned. In addition to the docs in this project, I know on the Microsoft side we'd also want to update this Microsoft Learn article and this blog post to mention the new commands.

I've also started an internal discussion with @craigloewen-msft and the WSL team in case anyone else with more subject mater expertise on WSL has feedback. My availability will be limited the rest of this week for US Thanksgiving.

[craigloewen-msft]

Changes also look good to me! My only request would be that for the removed commands, can we show an error output that this command has been removed and indicate to the user what command they should run instead?

Also @dorssel I will look at changing the blog post and docs to refer to the new commands. Please let me know once this is available and I will do so, and if you had any PRs or edits for that those are more than welcome. :) Thanks for the great description and changes here!

[andrewleech]

I agree, thanks heaps for the heads up @dorssel, the changes and rationalle make perfect sense to me and I think you've planned a great path forward. I've also had a number of discusstions / feature requests surrounding the "which WSL distro is being connected to" that I think this should help clarify, or at least simplify.

On a tangentially related note, I presume it's not possible / appropriate to have the service itself installed and running with enough priviledges to do the initial bind without needing an elevated user?

[dorssel]

@craigloewen-msft
Yes, I will change it such that executing anything starting with usbipd wsl ... will error with a link to the documentation.

FYI: currently the following links are already displayed to the user when something is wrong with WSL. These will remain (for the new usbipd attach --wsl) and I will add one for the old usbipd wsl:

• If WSL isn't installed at all: https://aka.ms/installwsl
• If the WSL distro is version 1: https://docs.microsoft.com/windows/wsl/basic-commands#set-wsl-version-to-1-or-2
• If the WSL distro doesn't have the client tools installed: https://github.com/dorssel/usbipd-win/wiki/WSL-support
• If there is a distro installed, but it is not set as the default (which happens if the user removes the default distro): https://docs.microsoft.com/windows/wsl/basic-commands#set-default-linux-distribution
• If WSL itself is installed, but there is not a single distro installed: https://docs.microsoft.com/windows/wsl/basic-commands#install

Note that these URLs probably require updating. doc to learn, for example. And (only) one of them uses a short link (aka.ms). And one is referring to my Wiki, whereas it is also on MS (I would prefer to have all WSL docs at MS).

[dorssel]

@andrewleech
I'll start a new discussion about the privilege/admin rights issues soon. Lets keep this discussion focused on the wsl command.

[dorssel]

@craigloewen-msft
How about this (current draft):

PS C:\> usbipd wsl anything --after the subcommand
usbipd: error: The 'wsl' subcommand has been removed; see https://learn.microsoft.com/en-us/windows/wsl/connect-usb#attach-a-usb-device on how to use the new syntax.
PS C:\>

(it's an error, so it will show in red; and on a terminal this will be word-wrapped like everything else...)

[dorssel]

@craigloewen-msft
Here is something new again:

I have managed to compile the required client-side tooling in a distro-independent way. It's already merged in the master branch; see the README.

This solves the long standing problem of users having to install the "right" client tooling, which is distribution dependent. It was especially cumbersome for Ubuntu, because even if you did install linux-tools-virtual (as explained in the docs), you had to enable it with an update-alternatives kludge. In fact, you had to repeat the kludge every time you updated the package. I myself have forgotten to do that more than once. It lead to by far the most reported issues.

How it was solved

usbipd-win (from version 4.0.0 on) will ship with a generic usbip tool. It will work on any distro, as it was statically linked and only depends on the kernel. And we know it will work on the WSL kernel, as that is under our (or rather: your) control. As a consequence, it no longer matters at all which distro is used for attach. You no longer have to install any client tools. This removes the need for any remarks like "how to prepare WSL for USBIP" or "install package X for distro Y". It will simply "just work(tm)".

More changes to come (TODO)

In the past, usbipd wsl attach would try to use the default distro (and assume the client tools were installed there), unless you explicitly used the --distribution option. You would get errors if there was no default distro, or if the default distro wasn't running, or if the default distribution was still WSL version 1. In all those cases you would have to supply the --distribution option to override the default. All of this just to select a distribution that happened to have the client tools installed.

Now that the required client tool will be shipped as part of usbipd-win, it really no longer matters what distribution is used to run the tool, as long as it is version 2 and running. Therefore, by default, the new usbipd attach --wsl command will simply use any running version 2 distro (preferring the default distro, if it can be used). This means, under normal circumstances, the user will never ever have to specify a distribution again. Either at least one version 2 distro is running, and it will just work. Or no version 2 distro is running (or installed) and the user will get an appropriate error message, as always.

However, I will leave the (optional) argument to usbipd attach --wsl [optional-distribution-name] in, just in case someone has a really odd/broken distro where even the statically linked usbip doesn't work. But I expect it will actually never be used.

I did consider using wsl --system instead of a distro, but that is not always available (e.g., if the user set guiApplications=false). So, this is not an option.

I did consider to use wsl --debug-shell, but that only allows one instance and won't work if it is already being used. Furthermore, the --auto-attach option is long-running; it would claim the one-and-only debug shell forever, So, this is not an option either.

[andrewleech]

Now that the required client tool will be shipped as part of usbipd-win, it really no longer matters what distribution is used to run the tool

That's a fantastic improvement, congrats on the great work!

[dorssel]

Everything is committed to the master branch now.

@craigloewen-msft
I've updated the links to:

    public const string AttachWslUrl = "https://learn.microsoft.com/windows/wsl/connect-usb#attach-a-usb-device";
    const string InstallWslUrl = "https://learn.microsoft.com/windows/wsl/install";
    const string ListDistributionsUrl = "https://learn.microsoft.com/windows/wsl/basic-commands#install";
    const string InstallDistributionUrl = "https://learn.microsoft.com/windows/wsl/basic-commands#install";
    const string SetWslVersionUrl = "https://learn.microsoft.com/windows/wsl/basic-commands#set-wsl-version-to-1-or-2";

I think those are right/stable.

Mirrored mode detection works great (it is using wslinfo --networking-mode). It is really nice to have usbipd attach --wsl working no matter which distro is running and not having to deal with installing additional packages.

It all seems to work nicely, except that there is a nasty bug in WSL mirrored mode networking. I've reported it at microsoft/WSL#10741. Either that needs to get fixed soon, or I will have to work around it by disabling keep-alive on all localhost connections. That has the drawback that people using localhost connections for tunneling do not have keep-alives either...

Regression testing starts now. ETA for 4.0.0 is about a week from now. Expect some documentation changes...

[dorssel]

@craigloewen-msft
Please have a look at https://github.com/dorssel/usbipd-win/wiki/README-4.0.0.
Suggestions/improvements are welcome.

Next up: I will modify the WSL Support wiki accordingly...

[dorssel]

@craigloewen-msft
Now also available: https://github.com/dorssel/usbipd-win/wiki/WSL-Support-4.0.0.