what to do about programs.sqlite

[cdepillabout]

As described in #1, we'd like to use ~/.nix-defexpr/channels/nixos-unstable/programs.sqlite to figure out what Nixpkgs package contains a given executable.

@jonascarpay asked:

programs.sqlite is actually provided by nix-channel, which is a little outdated at this point. Is there an easy way of manually generating a programs.sqlite for a certain commit?

I took a quick look at this, and here's what I've found so far:

• It looks like programs.sqlite is created by the files in https://github.com/NixOS/nixos-channel-scripts. In particular, it is created by https://github.com/NixOS/nixos-channel-scripts/blob/b2053e2a33926bc84541925affd3f46c53c59a07/generate-programs-index.cc, which is called by https://github.com/NixOS/nixos-channel-scripts/blob/b2053e2a33926bc84541925affd3f46c53c59a07/mirror-nixos-branch.pl.
• The above repo is somehow related to https://releases.nixos.org/nixos/unstable/nixos-22.11pre396557.5857574d459, which it either generates, or takes as input. In this directory, there is a packages.json.br file and a store-paths.xz file, which from the file name, sound like they could reasonably be used as inputs to generate programs.sqlite.
• It looks like there is a https://github.com/NixOS/nixpkgs/blob/e4bcf793713f281a9517f7097ad00e34873c42a5/pkgs/top-level/make-tarball.nix file in Nixpkgs that generates this packages.json.br file.
• There's a corresponding job on Hydra for this derivation. You can see it contains packages.json.br as an output available to Hydra:
  • https://hydra.nixos.org/eval/1773958?filter=tarball&compare=1773773&full=#tabs-still-succeed
  • https://hydra.nixos.org/build/185929175
• I'm not sure where store-paths.xz comes from.

[cdepillabout]

My suggestion would be: let's just pin binplz.dev to a single commit (whether that is unstable, or 22.05), manually download the programs.sqlite, and commit it to this repo. It's only 10MB. At some later point we could figure out a better solution.

[jonascarpay]

Sounds good to me! Let's leave this issue open until we have a long-term solution, but for now we just manually update it when we bump the nixpkgs pin.

[jonascarpay]

I looked into this some more today. It looks like command-not-found might get replaced by nix-index: NixOS/nixpkgs#39789. I'm seeing if we can just use that instead already.

[jonascarpay]

NixOS/nixos-channel-scripts#54 (comment)

[jonascarpay]

As mentioned in #21, we're leaving this open until we find a solution that doesn't require locally running nix flake lock --update-input nixos-channel on every new machine.

[kayhide]

How about making a dedicated repo which only contains programs.sqlite?
With it, we can consume desired version of programs.sqlite by just pointing commit hash or tag.

[jonascarpay]

That's essentially what I do for my nordvpn config here. I'll copy that and see if it's feasible here as well. I'm a little worried SQLite won't compress as nicely.

EDIT: SQLite files aren't compressed, and git should run compression on binary files, so I think we might be fine.

[jonascarpay]

https://github.com/jonascarpay/programs.sqlite/

[considerate]

Why wasn't this closed by #25?

[jonascarpay]

@considerate Oh, good catch, thanks