tree-wide: make rust jemalloc-sys use nixpkgs jemalloc build

[yu-re-ka]

Description of changes

Addresses #202863

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 23.11 Release Notes (or backporting 23.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

[yu-re-ka]

More difficult than I thought because some packages (where the unprefixed_jemalloc_on_supported_platforms feature is enabled) are not allowed to have the _rjem_ prefix depending on the platform.

[tjni]

What's your current thinking on how to address this? It seems hard to do automatically without better ways to override.

[yu-re-ka]

We can create a prefixed (as is currently in this PR) and only-prefixed-on-some-platforms version.
However for the only-prefixed-on-some-platforms version we need to exactly match the list here and hope that it doesn't change too much across versions of the (tikv-)jemallocator crates.

Then choosing the correct version of the hook should be a matter of trial-and-error.

Then another challenge is finding all rust packages that build with jemallocator

[tjni]

This does sound like the best option today.

However for the only-prefixed-on-some-platforms version we need to exactly match the list

Perhaps this can be considered an issue that upstreams would be amenable to fixing, so then our second variant is just the system jemalloc.

Then another challenge is finding all rust packages that build with jemallocator

I think the only way to find these right now is if someone like you encounters a breakage.

[alyssais]

The unprefixed variant doesn't seem to be used anywhere — is that still waiting for testing each package to determine which one it needs?

[alyssais]

I think we can maybe do

Suggested change

	export JEMALLOC_OVERRIDE="$1/lib/libjemalloc${stdenv.hostPlatform.extensions.library}"
	export JEMALLOC_OVERRIDE="@out@/lib/libjemalloc${stdenv.hostPlatform.extensions.library}"

and not need the if or the postFixup?

[yu-re-ka]

Indeed we can. done.

[hmenke]

This broke all Rust static builds that use jemalloc (which is pretty much everything). For example ruff

$ git checkout 799fff889b1e
$ nix-build -A pkgsStatic.ruff # error
$ git checkout HEAD^
$ nix-build -A pkgsStatic.ruff # succeeds

[alyssais]

Right, yes. We have two options, then. Make every package only conditionally depend on rust-jemalloc-sys when !stdenv.hostPlatform.isStatic, or set rust-jemalloc-sys to null in that case. @yu-re-ka, any thoughts?