Replace 'configurationsXyz' and 'modulesXyz' with 'configurations.xyz' and 'modules.xyz'

[gytis-ivaskevicius]

Simply put - the current approach is just not scalable, we got projects defining custom modules/configurations which are being recognized as 'unknown' due to the lack of a type system.
Some peeps suggest implementing nixpkgs options system or type system via JSON spec but I think the best solution here is to just remove the need for a type system in this case entirely.

Expected properties rename:
nixosConfigurations.hostname = {...} to configurations.nixos.hostname = {...}, so schema would be: configurations.<name>.<hostname> = {...}
nixosModules.moduleName = {...} to modules.nixos.moduleName = {...}, so schema would be: modules.<name>.<moduleName> = {...}
(Similar idea to REST naming convention and existing system-specific properties)

Also, may I add that there is one annoyance current nix flake check implementation:

nix/src/nix/flake.cc

Line 421 in 895dfc6

auto vToplevel = findAlongAttrPath(*state, "config.system.build.toplevel", bindings, v).first;

Alias in nixpkgs should be added so we could replace this line with something like this:
auto vToplevel = findAlongAttrPath(*state, "config.drv", bindings, v).first;

[edolstra]

It's not clear to me what the advantage of this proposal is. What's the usefulness of having a semantics-free configurations output type?

[gytis-ivaskevicius]

This allows the community to define its own module groups. Here are a few examples:

• nix-darwin users would use modules.darwin and confiugurations.darwin (currently they use darwinModules and darwinConfigurations attributes, which are 'unknown' and not checked by nix flake check)
• home-manager - same idea as nix-darwin
• Everyday nixers could make use out of such namespaces as well, for example having groups such as: modules.{server,desktop,cli,development,etc}. In fact I do have something similar implemented here https://github.com/gytis-ivaskevicius/nixfiles/blob/master/suites.nix (lines: 3, 19, 32)

[gytis-ivaskevicius]

Oh, and just to be clear

What's the usefulness of having a semantics-free

It is not semantics-free. Current design is nixosModules = <module>, this would change definition to modules.<name> = <module>

[bew]

It is not semantics-free. Current design is nixosModules = <module>, this would change definition to modules.<name> = <module>

But there may be other kinds of modules, nixos ones are not the only ones.
(Same for configuration)

[gytis-ivaskevicius]

But there may be other kinds of modules, nixos ones are not the only ones.

Yeah, which is why I am suggesting to make flakes schema more flexible, so it could accommodate nix-darwin/home-manager/etc modules as well.

[Pacman99]

Overall a good idea 👍 I think this would cleanup flake's relationship with configuration systems.

Largest question for me is whether nix flake check knows how to "check" home-manager/darwin systems? And if it doesn't I presume it would just check configurations.nixos.*.

Also to be clear it would be modules.<config-system-name>.<name> = <module> right?

Although, I could see modules actually following a pattern similar to packages here where a module can claim to support different config systems and it would actually be modules.<name>.<config-system-name> = <module>. So if modules.agenix has a nixos and darwin attribute, it can support bot those config systems.

[gytis-ivaskevicius]

Largest question for me is whether nix flake check knows how to "check" home-manager/darwin systems? And if it doesn't I presume it would just check configurations.nixos.*.

The answer is simple: Nope. One of the checks is the derivation path itself, there are a few options:

1. Create some sort of standard like instead of searching for config.something.toplevel just have a standard alias something along the lines of config.drv
2. Check for derivation only in case of NixOS configs, free pass for everything else 👎
3. Hardcode home manager/darwin projects drv paths into nix 👎

As for the last part - I did not quite get it. Assuming modules are identical I would expect something along the lines of this:

{self}:{
    modules.nixos.abc = import ./xyz.nix;
    modules.darwin = self.modules.nixos;
    
    # As of right now common approach is: https://github.com/gytis-ivaskevicius/flake-utils-plus/pull/111/files
    nixosModules = {abc = import ./xyz.nix;};
    darwinModules = self.nixosModules; # unknown properly
}

[marksisson]

I posted a copy of my flake (#6723) using nixosConfigurations attribute to contain home-manager and darwin configurations - this flake does not have errors when running nix flake check, but the extra complexity highlights the fact that the "configurations" abstraction could be better.

[roberth]

With a solution like https://github.com/hercules-ci/flake-parts you could remove the hardcoded schema checking in Nix and let the nixpkgs repo define nixosConfigurations, let nix-darwin define darwinConfigurations, etc. I'll be happy to remove the nixos* attributes from flake-parts.

I'm still surprised how much Nix is hardcoding ecosystem stuff that it has no business knowing about. It's as if we've forgotten that the nix implementation can't be pinned by expressions or lock files. Good thing flakes are experimental so this stuff can still be removed.

[DavHau]

I think getting this change in before the stable release of flakes is important.
More and more projects are evolving in the ecosystem that export modules, but there is no suitable output in the current schema.

Examples for these projects:

• flake.parts
• jupyenv (previously jupyterWith)
• drv-parts

And of course there are the well known:

• home-manager
• nix-darwin

In my opinion, the most reasonable schema would to be a 3 levels nested attrset, like:
modules.<config-system-name>.<name> = <module>.

@gytis-ivaskevicius Reading the OP it's not quite clear if you are suggesting a 2-levels or 3-levels nested attrset. Maybe you could edit the OP to make that more clear.

[gytis-ivaskevicius]

Updated

[roberth]

Earlier, we've decided not to check the NixOS modules with flakes anymore, because there was very little practical utility in doing so (as the things that might end up in the attribute tend to be indistinguishable from valid modules), along with it being somewhat flawed architecturally.

• flake check: allow modules to be paths #7356 (comment)

To get more use out of this, I think we need to first establish in the module system which things are definitely not modules. There's probably something to be gained there, by making it check against certain things that are known to be highly unlikely to be modules. Ultimately it is (educated) guesswork, so hardcoding that guesswork in Nix doesn't seem like the right thing to do. Together with the architectural argument, I think it'd be better to defer the checking to frameworks (flake.parts, etc), perhaps with a new interface for eval checks that frameworks can hook into. Nix's responsibility is then reduced to accepting and ignoring a modules output, documenting its existence, and letting the community evolve the right checks for it.

Similar logic would apply to the configurations attribute.

Superficially this might seem like a lazy-but-overengineered solution, but it is the one that allows community expressions to evolve in a forward compatible manner, noting the Nix's logic can not be pinned, unlike community expressions.

[roberth]

An example of evolution on the Nixpkgs side is

• module system: Improve error messages around faulty imports nixpkgs#197547

Examples of checking behavior that may evolve

• Adding a _type to the result of evalModules
• Rejecting _type in module imports and evalModules modules arguments
• Consequences of any ideas to make importing modules from flakes a bit more ergonomic (as discussed in the PR or something else). This one's more of an unknown unknowns example, fwiw.

[Atry]

Currently the module system in lib.modules is called NixOS Module, and the document is included in NixOS manual. However, there are more use cases of the module system other than NixOS, for example, https://github.com/hercules-ci/flake-parts .

I wonder if we could rename it to Nix Module and move the document to Nixpkgs manual.

[bew]

@Atry I think your message is quite orthogonal to this issue, I'd suggest opening a new issue for that instead

[Atry]

I agree documenting might belong to another issue, but it is still related to this issue, because this issue is about renaming nixosModules to modules.nixos, potentially allowing for modules other than NixOS Module.

[Atry]

I would argue if different "config system"s really exist. Most NixOS modules can be used as perSystem flake parts without any modification, e.g. hercules-ci/flake-parts#74 (comment)

For example, you can create a flake-parts project, then configure some Jupyter kernels using services.jupyter.kernels by importing "${nixpkgs}/nixos/modules/services/development/jupyter" to your perSystem module, then you can launch Jupyter in your project by turning systemd.services.jupyter.serviceConfig.ExecStart into a shell script.

After all, NixOS modules are just config file generators. We don't have to run the generated files in systemd.

[roberth]

@Atry, while this may be possible today, you should expect this to break, unless adequate testing is added in the nixpkgs repo to ensure that this use case keeps working. This essentially what RFC 78 is about. See my comment for an example of how the module system can be leveraged to avoid the unnecessary NixOS-specific complexity in your use case (although it leaves a lot to the imagination).

[roberth]

In this PR, the concept of "config system" or "module system application" is formalized as class. By omitting a _class declaration in your modules, you can keep them generic.
Using this term, we can define the current convention as

flake."${class}Configurations"
flake."${class}Modules"

This is also mentioned in the module system documentation (but the online manual hasn't updated yet).

The suggested attribute path convention would be:

flake.configurations.${class}
flake.modules.${class}

Furthermore the PR has added _type = "configuration"; to the values that are meant to go into configurations.${...}.

[roberth]

Neither naming scheme has a particularly good spot for generic modules.
In the module system, such modules are represented by _class = null; / class = null;, but nullModules or modules.null is unpleasant. In the existing convention, modules would be suitable, but this blocks off the potential for an unambiguous modules.${class} later.
Perhaps a better alternative is to replace null by "generic"? That way we can have genericModules or modules.generic without making the naming rules more complicated. I don't like that it slightly deteriorates the class checking data model though; nullOr str is most accurate.

[roberth]

Let me reply to @blaggacao here

More Context on home-manager

Home Manager users are fundamentally system-portable. Advancement here could trigger evolution there.

Definitions

If it's portable, then "system" is a parameter, which means it is not a configuration in the sense of the module system.
Or even in general. Looking at the wiktionary definitions for configuration, none of them suggest that there's any parameter that remains, and neither does it suggest that a configuration is not a singular thing.

It's unfortunate that user expectations and the bottom-up definition lead to an overloading of the term "configuration", and they meet in the flake schema.
Maybe it's helpful to have a term for what configurations.<system>.<name> would be; perhaps a configuration family, which we could define as a set of configurations that derive from roughly the same modules, but need to vary along a parameter, such as the host platform.

Simple schema

The practical problem with configuration families in relation to this issue's goal is that they have a different schema compared to a normal attribute set of configurations. If we were to put both in the configurations schema, we wouldn't succeed at making the schema more general, but rather we would have produced a set of renames.

It might still be possible to traverse such a heterogeneous attribute set without significant cost based on x._type or null == "configuration", but I wouldn't count on module system applications deferring their assertions to the config attribute alone, and even then it wouldn't be possible to traverse it generically without loading the flakes that implement these module system applications.

So in line with the goal of this issue, actually simplifying the schema, I'd rather keep configurations simple; just attrsOf configuration so to speak.
An application that needs something else can use its own attribute.

Modules instead?

As a final note, a different interpretation is that if a Home Manager configuration is truly portable, then "system" is not part of the configuration. Arguably the derivations it builds need to be configured, so there's a hole in that configuration. Such a hole, whether it's a function parameter or an option is representable using a module rather than a configuration.
Perhaps Home Manager should do the same thing that NixOps has to do. A NixOps configuration has many holes because of cloud resource state info, so it can only be represented by a module rather than a configuration. The configuration only exists within nixops' own specific evaluation.

This would look like:

modules.home-manager.foo = { ... }: {
  # ...
}

While I like the simplicity of this, it has some issues

• It's not clear which modules are "top level" ones that are suitable as the whole input to the module system application.
• specialArgs can't be declared. While I think specialArgs should be avoided as much as possible because it is truly non-modular "global state", it should be possible.
• It's hard to bootstrap a configuration from this: which lib to use? How is pkgs initialized?
  • applications will handle this differently

These issues seem sufficient to warrant a new flake output attribute containing this info.
The new flake output attribute would immediately solve the first issue. The others need to be handled by the attribute values.
I don't have a good suggestion for a name yet, so I'll just use partialConfigurations.

Brainstorming:

Possible attrs:

• partialConfigurations.<name>.modules: list of modules as usual; the flake's contribution to the module application's configuration.
• partialConfigurations.<name>.specialArgs: extra specialArgs that the module application should add
• partialConfigurations.<name>.lib: version of lib to use. I don't know if this is necessary or sufficient. Maybe more dependencies should be specified.
• partialConfigurations.<name>.application: version of the module system application to use (e.g. the home manager flake)
• partialConfigurations.<name>.applicationArgs: any arguments that module system application accepts. This could remove the need for a .lib attr

Actually .application suggests that we could standardize how module system applications are invoked. Seems a bit out of scope, and rather powerful. Too powerful?
I initially just forgot to add .<class> by mistake, but .application seems to fix that. The first two attributes seem necessary, but I have yet to make sense of the rest. Maybe just add .<class> to solve the problem at hand without getting carried away.
Needs more thought.

[blaggacao]

Without going too far, still:

perhaps a configuration family,

I could see definition of "configuration levels" helpful as a community-provided shared mental model.

This model captures different aspects (or "levels") of exogenous input, such as system, ip-addresses, identity, etc.

With the shared knowledge and understanding of such levels, we can make use of the two fundamental strategies of specialization in configuration management: functions (if there's a "hole") and overlays (if its a "variant").

The magic of (Nix') fix points is a really powerful primitive to build such an incremental/layered mental model while at the same time avoiding impracticalities (such as rendering the change of trunk values like system operationally and conceptually "cheap").

A shared mental model ("framework"), even without implementation, could help advance the Community's understanding and pave the way for better interfaces.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/flake-schemas-making-flake-outputs-extensible/32421/8