Use direnv environment

[diktomat]

Check for existing issues

• Completed

Describe the feature

Zed should support direnv, to enable using a different environment (espeacially $PATH) for different projects. In combination with #4978 this would enable e.g. using different versions of language servers and tooling, or tooling just installed for the project instead of system-wide using Nix.

If applicable, add mockups / screenshots to help present your vision of the feature

Inspiration from other editors:

• wbolster/emacs-direnv
• misuzu/direnv-subl
• direnv/direnv.vim
• direnv/direnv-vscode

[szlend]

A few issues that almost every editor suffers from when implementing direnv support:

• Direnv initializes too late, causing problems with certain extensions that rely on direnv to set the PATH (e.g. for utilities like language servers and linters). Ideally there was a way for the direnv extension to hook into some really early phase of initialization where it can set environmental variables before other extensions initialize.
• When you already have a project open and you open another project in a new window, the environment in the new window gets inherited from the first window. Normally this isn't a problem because it's just global system environment variables. But with direnv it's annoying because the second project will inherit environmental variables from the first project, potentially loading incorrect versions of linters, language servers, etc. Ideally the editor would save the state of environmental variables before applying direnv. And any new window that you open would inherit from that saved state.

Just something worth considering when designing the extension API and implementing this feature.

[misuzu]

• Direnv initializes too late, causing problems with certain extensions that rely on direnv to set the PATH (e.g. for utilities like language servers and linters). Ideally there was a way for the direnv extension to hook into some really early phase of initialization where it can set environmental variables before other extensions initialize.

Yep, the extension API must have support for this usecase to properly implement direnv integration. This is the reason why https://github.com/misuzu/direnv-subl is flawed - ST just doesn't have the necessary API. The direnv integration that just works would be The Feature that can ease the pain of setting up any project for many people.

[szlend]

Another useful feature that I think a direnv extension should have is to be able to re-load the environment so you don't have to restart your editor. The way this works in vscode for example is that it changes the process environment and then restarts the extension host (and all language servers as a result).

Also initializing direnv can take some time. For example if you use direnv to load a nix shell, it might actually start downloading packages or even compiling. So it might make sense to show some sort of indicator that direnv is still evaluating something. And a way to cancel it.

[jeremylightsmith]

This is definitely keeping us from using zed on our project. We use direnv + nix, and we can't use tasks w/o better support.

With VSCode, it's quite easy to use direnv, you just have to launch the editor from a directory that direnv has already run with code .. Would it be possible to have the same behavior with zed?

[jeremylightsmith]

it seems the termina doesn't maintain its env between runs, because if it did, we could work around this by sourcing a script like this:

#!/bin/bash

if [ -z "${DIRENV_INITIALIZED}" ]; then
  echo "Loading direnv..."
  eval "$(direnv export bash)"
  export DIRENV_INITIALIZED=1
else
  echo "direnv is already loaded."
fi

[mersinvald]

I found a workaround that is semi-usable: running zed with zed --foreground ./ after loading the environment in shell works, the editor plugins can access my nix shell environment, but it's far from ideal: this way you can only have one project/environment open at any time.

If I try to open a second project with a different env, with --foreground I get zed is already running error, and without the flag the environment is shared between all windows.

[mrnugget]

When you open a project, we spawn a login shell in that project's root dir, get the env from that process, and store it to be used with some language servers.

It sounds like you have direnv/nix setup so that would work, is the problem now that it doesn't work with tasks?

[jeremylightsmith]

Thanks @mersinvald , but your workaround doesn't work for me, specifically with tasks, which is my problem. I need to be able to run the test under my cursor from my IDE. And when I open up a task to do that, it doesn't have any of the env that direnv setup :(

[szlend]

With VSCode, it's quite easy to use direnv, you just have to launch the editor from a directory that direnv has already run with code .. Would it be possible to have the same behavior with zed?

This definitely does not work well in vscode, except for the initial window. The direnv vscode extension fixes this and makes it somewhat usable, but it still causes cross-project env pollution from the initial vscode window.

I think how Zed does it is a step in the right direction, though it's difficult to diagnose issues when something goes wrong. It might be a good idea to document the env loading behavior and some steps to help diagnose issues (e.g. how do I inspect the window's env, what login shell Zed used to initialize the env, etc.)

[mrnugget]

I think how Zed does it is a step in the right direction, though it's difficult to diagnose issues when something goes wrong. It might be a good idea to document the env loading behavior and some steps to help diagnose issues (e.g. how do I inspect the window's env, what login shell Zed used to initialize the env, etc.)

Right now that's all printed in the logs (zed: open logs). So when you open Zed, you should see that we do the same in the home directory.

But yes, documentation is a good idea.

Not sure how Zed figures out which shell to use to say if this is a problem or not though.

It uses SHELL and falls back to passwd lookup if SHELL isn't set.

It doesn't spawn nix shell. It uses your $SHELL.

This is what we do on start, in $HOME:

zed/crates/zed/src/main.rs

Lines 309 to 312 in 05b6581

	{
	load_shell_from_passwd().await.log_err();
	}
	load_login_shell_environment().await.log_err();

And this is what we do for the language servers, per project:

zed/crates/project/src/project.rs

Lines 11636 to 11665 in 05b6581

	// What we're doing here is to spawn a shell and then `cd` into
	// the project directory to get the env in there as if the user
	// `cd`'d into it. We do that because tools like direnv, asdf, ...
	// hook into `cd` and only set up the env after that.
	//
	// In certain shells we need to execute additional_command in order to
	// trigger the behavior of direnv, etc.
	//
	//
	// The `exit 0` is the result of hours of debugging, trying to find out
	// why running this command here, without `exit 0`, would mess
	// up signal process for our process so that `ctrl-c` doesn't work
	// anymore.
	//
	// We still don't know why `$SHELL -l -i -c '/usr/bin/env -0'` would
	// do that, but it does, and `exit 0` helps.
	let additional_command = PathBuf::from(&shell)
	.file_name()
	.and_then(|f| f.to_str())
	.and_then(|shell| match shell {
	"fish" => Some("emit fish_prompt;"),
	_ => None,
	});
	let command = format!(
	"cd '{}';{} printf '%s' {marker}; /usr/bin/env; exit 0;",
	dir.display(),
	additional_command.unwrap_or("")
	);

[musjj]

It looks like that rust-analyzer can now pick up the binary from the environment: #12418. So I added this to my config:

{
  "lsp": { "rust-analyzer": { "binary": { "path_lookup": true } } }
}

But it's still tries to download and run its own binary anyways (and failing because I'm on NixOS).

[mrnugget]

But it's still tries to download and run its own binary anyways (and failing because I'm on NixOS).

Yes, because that commit you linked to hasn't been in any release yet.

[mersinvald]

@mrnugget it doesn't work for me without the foreground hack. If I just open the project, the env from .envrc wouldn't be loaded for anything except the integrated shell.
With foreground, both plugins and tasks work fine, as zed is loaded within the direnv-initialized PTY.
I have cargo and rustup setup only in the flake shell, so it's easy enough to test.

When you open a project, we spawn a login shell in that project's root dir,

Is there a way to modify what runs in this shell before editor initialization?

[mrnugget]

@mrnugget it doesn't work for me without the foreground hack

what doesn't work? Only certain language servers make use of this environment right now.

[mersinvald]

@mrnugget it doesn't work for me without the foreground hack

what doesn't work? Only certain language servers make use of this environment right now.

I would like the editor to respect the directory env regardless of what specific plugin needs it. In my case, the issue is with rust-analyzer.

Log without the foreground hack (neither rustup or cargo are installed in the system, only in a project flake)

Support/Zed/languages/rust-analyzer/rust-analyzer-2024-06-11", working directory: "/Users/XXXXXX", args: []
2024-06-11T12:58:46+04:00 [ERROR] Language server rust-analyzer-2024-06-11 (id 1) status update: Failed to load workspaces.
2024-06-11T12:58:46+04:00 [ERROR] Language server rust-analyzer-2024-06-11 (id 1) status update: Failed to load workspaces.
2024-06-11T12:58:47+04:00 [INFO] Language server with id 0 sent unhandled notification LogMessage:
{
  "level": 0,
  "message": "[DEBUG] [agent] [2024-06-11T08:58:47.175Z] Telemetry initialized",
  "metadataStr": "[DEBUG] [agent] [2024-06-11T08:58:47.175Z]",
  "extra": [
    "Telemetry initialized"
  ]
}
2024-06-11T12:58:47+04:00 [INFO] reload git repository ".git"
2024-06-11T12:58:47+04:00 [INFO] reload git repository ".git"
2024-06-11T12:58:47+04:00 [INFO] add connection to peer
2024-06-11T12:58:47+04:00 [INFO] add_connection;
2024-06-11T12:58:47+04:00 [INFO] waiting for server hello
2024-06-11T12:58:47+04:00 [INFO] got server hello
2024-06-11T12:58:47+04:00 [INFO] set status to connected (connection id: ConnectionId { owner_id: 0, id: 0 }, peer id: PeerId { owner_id: 443, id: 2668270 })
2024-06-11T12:58:47+04:00 [INFO] set status on client 153260: Connected { peer_id: PeerId { owner_id: 443, id: 2668270 }, connection_id: ConnectionId { owner_id: 0, id: 0 } }
2024-06-11T12:58:50+04:00 [ERROR] crates/languages/src/rust.rs:443: Os { code: 2, kind: NotFound, message: "No such file or directory" }
2024-06-11T12:58:51+04:00 [WARN] request completed with error: request or operation took longer than the configured timeout time
2024-06-11T12:58:51+04:00 [ERROR] crates/client/src/telemetry.rs:492: request or operation took longer than the configured timeout time

Caused by:
    [28] Timeout was reached
2024-06-11T12:58:51+04:00 [WARN] request completed with error: request or operation took longer than the configured timeout time
2024-06-11T12:58:54+04:00 [ERROR] crates/languages/src/rust.rs:443: Os { code: 2, kind: NotFound, message: "No such file or directory" }
2024-06-11T12:58:55+04:00 [ERROR] crates/languages/src/rust.rs:443: Os { code: 2, kind: NotFound, message: "No such file or directory" }
2024-06-11T12:58:57+04:00 [ERROR] crates/languages/src/rust.rs:443: Os { code: 2, kind: NotFound, message: "No such file or directory" }

[mrnugget]

In my case, the issue is with rust-analyzer.

That should then be possible with the upcoming Preview release and when setting {"rust-analyzer": {"binary": {"path_lookup": true}}}.

[jeremylightsmith]

Alright, I want to show you where it's not working for me. You can see that when I go into my project directory, direnv runs and we are finding elixir from nix. But after launching zed, even with foreground, within a task, it finds elixir from asdf :(. Which means all my tests fail and I can't do things like mix test

For VSCode, it works because the first time I run a task, it opens and fails, then I can manually type direnv allow, and then future runs of the task use that same environment that has been direnv'd.

[mrnugget]

within a task

Yeah, that's not supported yet.

[srid]

Zed + direnv + Nix + Rust (rust-analyzer) has been consistently working well for me. Environment: macOS, with nothing (but Nix) globally installed.

The project in question: https://github.com/juspay/omnix

My Zed config:

{
  "base_keymap": "VSCode",
  "load_direnv": "shell_hook",
  "lsp": { "rust-analyzer": { "binary": { "path_lookup": true } } },
  "vim_mode": true,
}

[srid]

Incidentally, to get nixd (Nix language server) working, I had to add the following to Zed config:

{
  "lsp": {
    "rust-analyzer": { "binary": { "path_lookup": true } },
    "nix": { "binary": { "path_lookup": true } }
  },
}

Perhaps path_lookup should be the default for all language servers?

[srid]

One thing I noticed is this configuration works only if you put it in your user config (~/.config/zed/settings.json) but not the project config (./.zed/settings.json) -- as VSCode users do. That seems like a bug?

[rawkode]

Perhaps path_lookup should be the default for all language servers?

Yep! If the binary exists in the path, please just use it.

[mrnugget]

That seems like a bug?

Not strictly a bug, but: we only allow overwriting certain settings in project-specific settings. But yeah, agree, this should probably be one.

[mrnugget]

Yep! If the binary exists in the path, please just use it.

I agree, but last time I did that, lots of rust-analyzer stuff broke, I need to look into this again.

[rome-user]

@srid Which version of Zed are you using? I am using Zed 0.149.3 on Mac OS 14.6.1. My projects use Nix with direnv. Zed is able to find LSP servers like gopls, nixd, etc. in projects where they are specified in Nix flakes.

It was sufficient to set load_direnv to shell_hook in the settings.json.

[srid]

@rome-user 0.149.3 on macOS ARM 15.0 beta.

I definitely have to enable path_lookup for each LSP to get it to work (can confirm for rust and nixd).

[domenkozar]

A good step forward would be to have a global path_lookup flag that each LSP implementation needs to respect.

[DrewBurkhart]

In case it helps anyone else, this was my solution/workaround.

---

tl;dr

What was tripping me up is that it was saying Hit 4 reinstallation attempts for "rust-analyzer" (which I now realize I focused too heavily on as the final log message) but I think it was really 4 failed attempts to start rust-analyzer. Once I installed it via rustup, it worked just fine. That step wasn't required in VS Code so I just hadn't expected to need it. Not sure if that's expected or not.

---

My Problem when using nix + direnv + rust-analyzer (essentially just a restatement of earlier messages here)

In various projects that work as is in VS code with no RA-related settings, I'm having trouble getting RA to run. I've added

{
  "load_direnv": "shell_hook",
  "lsp": {
    "rust-analyzer": {
      "binary": { "path_lookup": true }
    }
  }
}

to ~/.config/zed/settings.json but it appears as though one of two things happens depending on how I start Zed (which matches what was explained above):

1. If I open via CLI (zed .) it doesn't load the env in the terminal and rust-analyzer seems to start ok but then errors with "failed to load workspace" in the logs (with no other apparent context).

2. If I open via spotlight it loads the env correctly but the logs indicate that it's failing to install rust-analyzer repeatedly (same as @domenkozar noted above)

[INFO] found user-installed language server for Rust. path: "/nix/store/dlipzjnxcypn8sz8kgaqn7hw26j4shki-rustup-1.26.0/bin/rust-analyzer", arguments: []
[INFO] starting language server. binary path: "/nix/store/dlipzjnxcypn8sz8kgaqn7hw26j4shki-rustup-1.26.0/bin/rust-analyzer", working directory: "/Users/me/Development/proj", args: []
[ERROR] cannot read LSP message headers
[ERROR] failed to start language server "rust-analyzer": oneshot canceled
[ERROR] server stderr: Some("error: 'rust-analyzer' is not installed for the toolchain '1.80.0-aarch64-apple-darwin'\n")
[ERROR] Hit 4 reinstallation attempts for "rust-analyzer"

This is the same behavior as when I follow the steps @mrnugget mentioned above

You: instead of doing zed . you could do cd .. && zed my-dir, so that direnv is unloaded, the env vars removed and direnv is properly loaded.
You: you can run zed --foreground . too, which should fix the issue. But I don't consider that a fix, more like a workaround.

---

My Solution/Workaround

Basically I eventually just followed the error I'd overlooked in the logs which states

[ERROR] server stderr: Some("error: 'rust-analyzer' is not installed for the toolchain '1.80.0-aarch64-apple-darwin'\n")

So I ran

rustup component add rust-analyzer --toolchain 1.80.0-aarch64-apple-darwin

and then ran

cd .. && zed proj

Works great now!

[deng232]

in nixos. when I launch zed inside an manually launched development shell. direnv work properly. but with terminal direnv shell hook setup, zed termianl shows getting inside development shell but it is not develop environment. both "shell_hook" and "direct" setting behave the same. however going to directory using workspace:open has no problem.

[mrnugget]

@DrewBurkhart thanks so much for that debugging! That's going to help.

[domenkozar]

Currently, some LSPs call workspace.which(binary) to get the binary if it exists, while others use path_lookup settings (which is off by default).

Ideally, the extension API would provide find_executable(name) function and then have a global setting for path_lookup configurability for uniform experience.

Once that lands I'll write documentation for devenv.

[mrnugget]

Currently, some LSPs call workspace.which(binary) to get the binary if it exists, while others use path_lookup settings (which is off by default).

I'm in the process of turning path_lookup for rust-analyzer on by default. #17926

[xuoe]

I no longer experience this issue with the latest version (0.164.2). Previously, Zed wouldn't pick up project-specific LSP binaries; it would prefer either global/user ones or, if those were missing, those tied to extensions (installed in $XDG_DATA_HOME). Tried most things people here recommended to no avail.

Now it loads the expected, project-specific, binary (here templ) even if Zed is started from a .desktop entry:

[INFO] starting language server process. binary path: "/nix/store/5xdjrsqz69k324826bkn4514bhxqgzf9-templ/bin/templ", working directory: "/home/me/my/project", args: ["lsp"]

Skimming the release notes, I suspect it may have something to do with #21039.

---

Unfortunately, it seems to work correctly only when reloading a project. If Zed auto-loads a project on startup, it fails to detect the binaries, but if I toggle between projects, it detects the binaries both in the secondary project and the initial one.

In addition to that, this correlates with the shell (fish) and/or terminal pane not working correctly: deleting characters (or whole words) deletes them only logically, but not physically (they are still displayed); my configured shell prompt colors are also missing.

One peculiar thing is how the terminal tab handle differs in these two scenarios. Notice the Zed-inserted horizontal divider after the inserted fish cwd prompt:

Preloaded	Reloaded

[mrnugget]

Unfortunately, it seems to work correctly only when reloading a project. If Zed auto-loads a project on startup, it fails to detect the binaries, but if I toggle between projects, it detects the binaries both in the secondary project and the initial one.

Is it possible that you start Zed from the CLI there? In that case, that environment takes precedence over everything else we load.

[xuoe]

Is it possible that you start Zed from the CLI there? In that case, that environment takes precedence over everything else we load.

Yes. The result of doing that is the same as opening a project from an existing instance.

Here are the scenarios, possibly non-exhaustive, where the LSP binaries are un/detected:

Undetected	Detected
$ zed my/project	$ cd my/project && zed .
$ zed and waiting for Zed to auto-load the last project	$ zed --new and opening a project
similarly, launching Zed from an application launcher, where Zed auto-loads the last project	$ zed and opening a project, regardless of one being auto-loaded on startup

After comparing Zed logs and shell environments across all these scenarios, I've concluded that, in the Undetected cases, the main Zed process simply doesn't load the direnv environment. However, in both Undetected and Detected cases, the Zed terminal shell is spawned with the correct environment. Toggling between shell_hook and direct had no apparent effect.

Relevant log lines with the actual paths elided:

[INFO] set environment variables from shell:/run/current-system/sw/bin/fish, path: ...
[INFO] using project environment variables from CLI. PATH=...

The paths in question, present in all the Detected cases above, yet missing in all the Undetected ones, are as follows:

• <project>/.direnv/bin
• project-specific /nix/store/* paths generated after flake.nix is applied

---

Looking at the code, I'm not sure why a login shell must be invoked just to grab the environment. To my mind, it is the user's responsibility to set their session variables as they see fit.

In other words, the environment available to Zed at invocation time, regardless of how it's invoked, that should be the initial environment. The environment generated by direnv export json in the project directory is the environment diff that should be applied to the initial one and passed to terminal shells as well^[1]. This would streamline the process and make it easier to reason about.

^[1] For the sake of completeness (and correctness), this would likely have to be bidirectional (i.e., terminal shell -> Zed should work as well), in the event that direnv picks up changes to .envrc or flake.nix; this latter step would possibly require LSP server path checks and restarts, if paths differ. Tasks would likely also be affected.

[szlend]

Looking at the code, I'm not sure why a login shell must be invoked just to grab the environment. To my mind, it is the user's responsibility to set their session variables as they see fit.

My understanding is this is necessary to get a fresh base environment, otherwise the environment from the first window pollutes the environment of subsequent windows you open, since they all share the same base environment (they're not fresh processes but fork of the original process).

In other words, the environment available to Zed at invocation time, regardless of how it's invoked, that should be the initial environment. The environment generated by direnv export json in the project directory is the environment diff that should be applied to the initial one and passed to terminal shells as well[1]. This would streamline the process and make it easier to reason about.

This has caused me countless problems in vscode. I would open project A in vscode, and it would set some env var like NIX_DONT_SET_RPATH=1. I would then open another project B, and that project would suddenly build incorrectly because of cross-environment pollution.

Imo this is one of the things zed does right, even if it doesn't yet handle every edge case. Ideally each zed window was a fresh process, but I'm not sure how realistic that is given the IPC needs.

[uncenter]

Is there a reason this issue is still open? Seems like support is working fine now for the most part, other issues should probably be bug reports since the core functionality should be all there? I got confused thinking Zed didn't support direnv at all because this feature request was open.

[mrnugget]

Yeah, agree. Let's close this and open separate issues with MREs in them.