Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use binary cache at store_uri for certain operations #1360

Open
wants to merge 4 commits into
base: nix-next
Choose a base branch
from
Open

Use binary cache at store_uri for certain operations #1360

wants to merge 4 commits into from

Commits on Feb 12, 2024

  1. Use BINARY_CACHE_STORE where appropriate 

    Replaces / Closes NixOS#1352

Consider the following setup:

* `store_uri` points to a "special" store, e.g. `s3://` or `file://`.
* Hydra runs inside e.g. an nspawn container and exclusively builds
  on remote systems.

Then, build products are never in the nspawn's `/nix/store`. This in
turn means that `$MACHINE_LOCAL_STORE->isValidPath` is always `false`
(or `0` 🤷) for output paths and thus Hydra wrongly claims that the
output got garbage collected.

Instead, use the BINARY_CACHE_STORE to look for the availability in the
correct store.

Everything touching the `drv` rather than the output paths still uses
MACHINE_LOCAL_STORE: this is because `hydra-eval-jobs` does `openStore()`
on `auto`, so the derivations end up there rather than on the
BINARY_CACHE_STORE.
    @Ma27
    Ma27 committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    507bd9c View commit details
    Browse the repository at this point in the history

  2. Always check for build availability 

    Now that we don't assume the local store anymore, but use the
BINARY_CACHE_STORE which is effectively `openStore(store_uri)`, it
doesn't really matter where the store is.

Also, this doesn't seem too expensive for at most 5 build outputs.
    @Ma27
    Ma27 committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    44c98e1 View commit details
    Browse the repository at this point in the history

  3. Use subs instead of global variables for local & binary cache store 

    Otherwise the `include`-test would fail: this test defines a `hydra.conf`
with an `include` statement. However, the file to be included is created
after the test base is set up. This means that - among others - the
`Hydra::Helper::Nix` module is loaded before the file to include exists.
Since the global variables are directly initialized and try to read from
the config, the test fails, because loading the config at that time
doesn't work. Delaying this into a subrouting solves the issue.

I decided against messing around with the config mechanism here since I
consider this behavior to be correct.

The local store gets (i.e. `Hydra::Store->new()` w/o args) cached in the
Perl bindings already[1], however this isn't the case for the binary
cache store. Because of that, a `state` variable is used to cache the
binary cache store. At runtime, the location of that store doesn't
change anyways.

[1] https://github.com/NixOS/nix/blob/e3ccb7e42a55529d8d50544e7e0cebb5cbc606cf/perl/lib/Nix/Store.xs#L88-L96
    @Ma27
    Ma27 committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    67de7a3 View commit details
    Browse the repository at this point in the history

  4. Provide store for getDependencyGraph 

    This method is applicable on all kinds of stores, so let's
give it a store rather than letting it decide which store to
use.
    @Ma27
    Ma27 committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    e69d857 View commit details
    Browse the repository at this point in the history