PoC modules

[arnaud-lb]

This pull request is a proof-of-concept inspired by the Modules RFC Spec Brainstorm, with the additional goal of providing the compiler with a broader view of the codebase (encompassing the entire module tree).

The primary objective of this PoC is to evaluate whether modules can be implemented with reasonable performance despite this goal.

TL;DR:

• Additional constraints have been added to how modules must be structured.
• The Symfony Demo application is 12% faster in production settings after conversion to modules but slightly slower in development settings (and much slower on a cold cache).
• The current implementation does not yet benefit the optimizer. Making the optimizer module-aware could provide an additional 3–10% performance boost. (Compile-time function resolution alone yields a 2-4% boost.)

Additional constraints

The following constraints were added (some are in the spec, but not all) :

• Modules are only loaded for their class and function declarations. Other effects are discarded, and conditional declarations are persistent until the module cache is invalidated. Combined with other constraints, this makes module loading fast.
  Note: Modules being pure apart from class and function declarations is not enforced (yet) for uncached modules. We could enforce it by restricting which expressions can be used in top level statements, which functions can be called, delaying error handling, and delaying output (because of buffering handlers).
• A module can only depend on classes or functions declared in the same module or an other module, but not in a non-module file
• Circular dependencies between modules are not allowed (circular dependencies between files of a single module are allowed)
• All module dependencies must exist or must be autoloadable. Combined with timestamp validation and the other constraints, this gives the compiler a view of the entire module tree.

Behavior

1. Loading Modules:

• Modules can be loaded by calling require_modules($descriptors), with $descriptors an array of module.ini file paths (format detailed below)
• require_modules() can be called in an autoloader
• require_modules() only has an effect on the current request

4. Uncached Module Behavior:

• When loading a module for the first time, all files matched by the module.ini are executed
• Dependencies are eagerly autoloaded
• Four sets of elements are cached:
  • The classes declared by the module
  • The functions declared by the module
  • The modules it depends on
  • The timestamp of all executed files (for cache invalidation)

5. Cached Module Behavior:

• The module dependencies are loaded first.
• If opcache.validate_timestamps is enabled, we check whether the module files have changed, or if files was added or removed
• Cached classes and functions are declared (added to EG(class_map) / EG(function_map)). This is a cheap operation: classes are already in memory, and are already linked
• No files are executed

Other details:

• If a module file does not have a module statement, this is an error
• If a module file has a module statement with a different name than the one declared in module.ini, this is an error
• Trying to load a module that is already loaded is an error (if we have loaded module Foo, calling require_modules(["module.ini"]) is an error if module.ini declares module Foo)

module.ini format

The module.ini format is similar to what is described in https://github.com/Crell/php-rfcs/blob/master/modules/spec-brainstorm.md, but some compromises were made for ease of implementation, for now.

The file accepts the following entries:

• module: Defines the module's namespace
• files: A space-separated list of fnmatch() patterns. Patterns are anchored to and relative to the directory containing module.ini. Unlike in glob() patterns, * matches / in fnmatch() patterns, so src/*.php matches .php files recursively in src/.
• exclude: A space-separated list of fnmatch() patterns. Anchored to and relative to the module.ini file directory.

Example:

module=Foo\Bar
files=*.php
exclude=Test/* Resources/*

Dependencies

Dependencies are all classes and functions used by a module. This includes implemented interfaces, parent classes, parameter types, return types, property types.

Classes referenced in instanceof, or in ::class, are not considered dependencies.

Classes and functions defined in a file, but not declared during the first execution, are not inspected, and are allowed to have unmet dependencies. In the following example, if class Dependency does not exist, neither does class C, so this file doesn't have unmet dependencies:

module Test;

if (class_exists(Dependency::class)) {
    class C extends Dependency {}
}

Optional dependencies

Packages with optional dependencies may either use conditional declaration like the example above, to prevent errors during module loading, or move the dependent code in a separate module (potentially in a sub-namespace).

For example, the symfony/cache package may have a sub-module for each storage adapter.

Conditional instantiation of optional dependencies is not really possible currently. E.g., this code will result in an error at compile time if Dependency does not exist:

if (class_exists(Dependency::class)) {
    $obj = new Dependency();
}

It may be possible to improve this.

Classes whom only some methods have an optional dependency (e.g. via a parameter type) must be split.

Execution order

During the first load, we compile all files to generate and install a class map autoloader. After that, we execute all files in alphabetical order, relying on the autoloader to satisfy any dependency.

Executing files in topological order would have been a possibility, but an autoloader is still required to resolve circular dependencies between classes during linking. For example, the following files have circular dependencies that can only be resolved by autoloading:

// a.php

interface I {
    public function f(): A;
}

class A implements I {
    function f(): B {
        return new B();
    }
}

// b.php

class B extends A {
}

Instead of using a class map, it would be possible to just execute the remaining module files in the autoloader, since these are going to be executed anyway. This would result in a simpler implementation.

Internal classes

Currently, the implementation requires that all classes can be linked and cached in opcache. Unfortunately some classes can not be cached in their linked state (e.g. enums, and classes extending internal classes, at least on Windows). The PoC hacks around this, but a proper implementation would need to make these classes, or to relax this requirement (at the cost of performance).

Performance

Performance was measured on the Symfony Demo application after converting its vendor packages to modules. As the conversion required some tweaking ("fixing" circular dependencies, disabling some dependencies), I applied the same changes to the baseline.

Wall time (cached):

	php-cgi -T10,10 REQUEST_URI=/en/blog index.php
	validate_timestamps=0	validate_timestamps=1
baseline	baseline	baseline
modules	-12.42%	+2.55%

Modularized Symfony Demo is ~12% faster in production settings and ~3% slower in dev settings (due to the larger number of files to validate).

The same benchmark on an old laptop shows similar improvements in prod settings, but worth regression in dev settings:

Expand old laptop results

	php-cgi -T10,10 REQUEST_URI=/en/blog index.php
	validate_timestamps=0	validate_timestamps=1
baseline	baseline	baseline
modules	-13.44%	+28.13%

Wall time (cold):

	php-cgi -T0,1 REQUEST_URI=/en/blog index.php		php bin/console
	no cache	file cache	no cache	file cache
baseline	baseline (0.3336s)		baseline (0.0986s)
modules	+323.26% (1.4119s)	-41.10% (0.1965s)	+1289.93% (1.3699s)	+86.44% (0.1838s)

In this benchmark, we measure the average time of a single web request on /en/blog, or a single CLI execution of bin/console. Both are considerably slower with modules: there are more files to compile, and the cache is cold. This is critical for the CLI as the cache is always cold.

Using a file cache (warmed up before the benchmark) mitigates this entirely for the web benchmark. The CLI is still slower, but orders of magnitude less, and mostly because the baseline is very fast.

We should consider loading the opcache extension by default in all SAPIs, and enabling the file cache by default in CLI.

Memory usage:

• When cached: Peak memory usage does not change in meaningful ways once modules are cached.
• When not cached: During the first request, when modules are not cached, the memory usage increased considerably (from 5MiB to 19MiB: +280%).

Number of declared classes:

• 3974 with modules
• 857 without modules

SHM usage: +39%

Performance regressions

As seen above, modules are a few % slower in development settings (opcache.validate_timestamps=1), due to the larger amount of files to validate, and much slower during the first request (on a cold cache), due to the larger amount of files to compile.

The cold cache slowdown may be an issue especially for CLI scripts, as the cache is always cold. This may be mitigated by enabling the file cache by default in CLI.

The timestamp validation overhead may old be mitigated in a few ways:

• Let composer manage the invalidation of vendor packages, so that vendor packages do not have to be validated (e.g. check only the module.ini file, and ensure that composer updates its timestamp when the package version changes). One drawback is that it makes it more difficult to hack / make changes to vendor packages. Maybe we can use a different behavior depending on whether a package is installed from source or from tarballs.
• Use inotify or similar mechanisms. This would eliminate the overhead entirely. One drawback is that this may require some additional configuration from the user, depending on the system and the number of files.

Local visibility

[Not implemented]

Not implemented yet, but this could be implemented with zero runtime cost, like this:

• Local symbols are mangled in the symbol table, so that looking up a local class yields nothing. As a result, normal class lookup is unchanged, and no overhead is added.
• Symbols referenced in a module are resolved at compile time, with local-visibility-aware lookup code.

Modules chunks

[Not implemented]

Files with a module statement can not be required/included, so it is not possible to "sneak" a class in a foreign module. But also, modules can not have lazy-loaded / optionally-loaded files currently.

It may be desirable lazy-loaded / optionally-loaded files may be desirable to, at least for tests: Test files should not be loaded most of the times, except when running tests, but tests must have access to local-visible symbols.

One way to allow chunks of modules to be loaded later, and optionally, would be to declare "module chunks" in the ini descriptor:

module=Foo\Bar
files=*.php
exclude=Test/
[tests]
files=Tests/*.php

require_module(array $descriptors, string $chunk);

In the example above, the module Foo\Bar doesn't load tests by default, but tests can be loaded by calling require_module('module.ini', 'tests');. Tests are in the same namespace as the rest of the module, and have access to package-visible symbols.

TODO

• The PoC is implemented in opcache for simplicity, but a proper implementation would work without opcache
• The compiler doesn't make profit of this, yet
• Compile-time function resolution
• Constant support

[Seldaek]

Very cool, thanks for all the work @arnaud-lb. Some comments:

Modules are only loaded for their class and function declarations.

This sounds to me fine, but maybe it'd be good if modules could have an init.php or smth that lets the module run whatever impure stuff it needs to run. That file would then be scheduled to be included after modules are loaded?

A module can only depend on classes or functions declared in the same module or an other module, but not in a non-module file

That sounds like it'll slow down adoption as it essentially makes it a big game of chicken and egg where every dependency has to be modularized first.. But it's ok I guess 🙂

Circular dependencies between modules are not allowed

Sounds like a difficult restriction to keep. It might be fine in isolation, but combined with the above restriction you end up with the issue that if something has circular dependencies, it is not modularizable, and thus anything using it is also not modularizable, which sucks. If any restriction needs to be lifted I'd say it's this one.

Optional dependencies

this also sounds very restrictive to me that you cannot have a parameter type for a class that does not exist without causing issues.. Why is that so? I imagine you could do it like for local symbols, keep the param type hint as a mangled version if the dependency does not exist, and if anything is passed in then it'd fail.. but if it isn't used all good? Same for conditional new perhaps, where as long as the class is not used/instantiated it would be fine?

Performance

That sounds very promising already, obviously the dev downside is unfortunate but sounds like we can live with it, and maybe it could be improved by having some dev mode in php that does not include/validate all files at once in a module but rather just sets autoloaders per module and lets things be loaded as needed? That might be too risky I guess, as it probably wouldn't be able to validate things in a way that guarantee the module would then still work in prod mode

Module chunks

Smart idea I think, for tests we wouldn't even need the optimizations to be applied across the entire set of module chunks, each chunk being loaded/optimized as its own piece would be enough.

[arnaud-lb]

Modules are only loaded for their class and function declarations.

This sounds to me fine, but maybe it'd be good if modules could have an init.php or smth that lets the module run whatever impure stuff it needs to run. That file would then be scheduled to be included after modules are loaded?

This seems totally doable

Circular dependencies between modules are not allowed

Sounds like a difficult restriction to keep. It might be fine in isolation, but combined with the above restriction you end up with the issue that if something has circular dependencies, it is not modularizable, and thus anything using it is also not modularizable, which sucks. If any restriction needs to be lifted I'd say it's this one.

Ok, thank you for the feedback. I will see if we can lift this one.

Optional dependencies

this also sounds very restrictive to me that you cannot have a parameter type for a class that does not exist without causing issues.. Why is that so?

One reason is that static analysis would need the definition of all symbols, including parameter types.

An other reason is that in order to make module loading fast, we have to link classes before caching them, which may require the know the definition of parameter and return types as well. Although this is not the case of existing code with optional dependency as parameter type, otherwise such code would not work. So if we abandon static analysis, we can probably lift this restriction in cases that work today.

If we implemented multi-version support, being able to resolve all symbols at compile time would also make things simpler.

I imagine you could do it like for local symbols, keep the param type hint as a mangled version if the dependency does not exist, and if anything is passed in then it'd fail.. but if it isn't used all good? Same for conditional new perhaps, where as long as the class is not used/instantiated it would be fine?

This could work indeed. I will think about this. One possible drawback is what happens if the symbol is declared later.

Performance

That sounds very promising already, obviously the dev downside is unfortunate but sounds like we can live with it, and maybe it could be improved by having some dev mode in php that does not include/validate all files at once in a module but rather just sets autoloaders per module and lets things be loaded as needed? That might be too risky I guess, as it probably wouldn't be able to validate things in a way that guarantee the module would then still work in prod mode

Yes, this would increase the risk of having divergent behaviors between dev and prod.

[IMSoP]

Regarding the dev mode performance, would it make sense in terms of separation of concerns to have an additional mode opcache.validate_timestamps=per_module, which only checked the timestamp of the module.ini file. Or maybe even just a function opcache_invalidate_module($module_ini_path)?

Then userland scripts could provide multiple strategies for invalidating the module:

• Composer could invalidate modules it has installed
• Frameworks which already have a build/cache-clear step could call it for the main module
• A "watch" script could parse a module.ini and install appropriate inotify hooks or whatever the OS provides
• Maybe some IDE / dev container systems would build it into the editing experience

[arnaud-lb]

Agreed. I suggested something very similar in the "Performance regression" section :)

Maybe this could be controlled by a require_modules() parameter, so the composer autoloader could enable module.ini-only validation depending on the module being loaded. E.g. enable that for vendor packages only, but not vendor packages installed from source.

It's possible to watch files efficiently with inotify and similar APIs, and I think it's possible without a separate worker. However these APIs limit the number of watched items, so it may not work out of the box depending on os and project size.

[Crell]

Very cool, Arnaud! I love it. Though of course I do have comments. :-)

The big one is that I don't think referring to validate_timestamps=1 as "dev mode" is entirely accurate. A number of systems will do code generation at runtime, out of necessity. Eg, any application that allows runtime enabling/disabling of a user module/extension/plugin may need to regenerate its compiled container. (Drupal still does this, I believe.) So the performance regressions under validate_timestamps=1 will sometimes be prod regressions, and we need to account for that. (validate_timestamps=0 being faster is fine and expected, regardless.)

As you note, it's also a problem for the CLI. Where would the cache even go for a CLI command, though?

I suppose to some extent that could be mitigated by smart module design; eg, many packages may want to be split into multiple modules to minimize the amount of excess/unused code that gets loaded. But that could reduce the effectiveness of any compiler optimizations, too. Where the "right" place is to chop up a package into modules is going to be an open question.

Though, the reason require_modules() takes an array of module files was to allow front-loading multiple modules into a single opcache segment, so it could be all optimized together. So either a package author or app author/administrator could load multiple related modules at once, even from the same package. (Eg, you know you're using the Valkey driver for cache, so just require_modules(['symfony/cache/module.ini', 'symfony/cache/valkey/module.ini']) to get them both optimized together. (How feasible that is remains to be seen.)

I like Jordi's suggestion of init files per module. Rather than a hard-coded filename, though, could it just be another entry in the module.ini file? (Or whatever that evolves into, since I'm not a huge ini fan.)

I really like the module chunks idea, which could also potentially help with minimizing the excess loading. Note that for some systems this would require changing their test namespace; Various conventions include putting Test in the namespace of test classes somewhere. I don't do that myself, but I've seen plenty of people that do. That code would need to be refactored to make it module-able. That could be a lot of busy work, but probably automatable, I'm guessing.

It would probably be prudent for FIG to have a PSR on common chunk names and usage patterns.

The way local visibility is described, would it work for properties and methods as well, or only classes/functions? We presumably want both. (Though as long as we're confident it could be done, visibility is a logical break point to split off to another RFC if we wanted to.)

If I read correctly, there's something about Enums that makes them... incompatible with modules? Hackliy supported? I don't quite follow the limitation there or what we can do about it.

My original writeup never came up with a name for "a unit of compilation." It's still just "X". :-) Any suggestions on what that should be called, if it's even relevant?

I'm not clear if this part of the brainstorm is viable:

If the file declares a module that is already loaded AND is it not a child or sibling of the module.ini file that was used to load it, it is a fatal error. This allows the package to have module files that are not bulk-loaded (because they are rarely used), but prevents other packages from "sneaking" into a foreign module.

I don't know if this is fully handled by the chunks concept. Maybe it is. Thoughts?

All in all, the top line performance numbers (~10% performance boost already, and the potential for another ~10%) make this extremely interesting and valuable to pursue. The primary concern is ensuring we do not hurt any valid common workflows along the way.

[derickr]

Additionally, I read "Local symbols are mangled in the symbol table, so that looking up a local class yields nothing. As a result, normal class lookup is unchanged, and no overhead is added." — how are you proposing the mangled form looks like?

[derickr]

Would having the name module here, and also as internal.module where it does something totally different, not be confusing?

[arnaud-lb]

Yes, I agree. I've since switched to the name user_module in most places, to avoid confusion with modules (which refer to extensions), but forgot to rename this one.

Please keep in mind that the code here is very PoC-quality, and definitely needs some cleanup.

[derickr]

Does this mean that modules only work with opcache installed?

[arnaud-lb]

That's right. As mentioned in the PR description, this is currently implemented in opcache for simplicity, but a proper implementation would move that somewhere else.

However, modules without are cache a slow to load (as seen in the "Wall time (cold)" benchmark results), and I would not recommend to use them without opcache.

[arnaud-lb]

@Crell thank you for the feedback!

The big one is that I don't think referring to validate_timestamps=1 as "dev mode" is entirely accurate. A number of systems will do code generation at runtime, out of necessity. Eg, any application that allows runtime enabling/disabling of a user module/extension/plugin may need to regenerate its compiled container. (Drupal still does this, I believe.) So the performance regressions under validate_timestamps=1 will sometimes be prod regressions, and we need to account for that. (validate_timestamps=0 being faster is fine and expected, regardless.)

I see. Unfortunately, right now I don't see a way to make modules as fast as baseline with full timestamp validation enabled. I will think about it.

Such systems could support validate_timestamps=0 by explicitly telling opcache to invalidate a module, with an equivalent of opcache_invalidate(). Or they could rely on module.ini-only validation.

As you note, it's also a problem for the CLI. Where would the cache even go for a CLI command, though?

For CLI, I was thinking of relying on the opcache file_cache to reduce compilation overhead. However, there are two issues with this: Currently, the file_cache is not considered stable, and it's not enabled by default (neither is opcache). If we can stabilize the file_cache, enabling opcache.file_cache by default (with opcache.file_cache_only) in CLI would be a nice improvement event without modules.

Testing CLI performance with the file_cache is the next item in my TODO.

I suppose to some extent that could be mitigated by smart module design; eg, many packages may want to be split into multiple modules to minimize the amount of excess/unused code that gets loaded

Agreed. Ideally, applications should split by feature/use-case. However, complex frameworks may quickly pull a lot of modules anyway.

[...] split into multiple modules to minimize the amount of excess/unused code that gets loaded. But that could reduce the effectiveness of any compiler optimizations, too. [...]

Though, the reason require_modules() takes an array of module files was to allow front-loading multiple modules into a single opcache segment, so it could be all optimized together. So either a package author or app author/administrator could load multiple related modules at once, even from the same package. (Eg, you know you're using the Valkey driver for cache, so just require_modules(['symfony/cache/module.ini', 'symfony/cache/valkey/module.ini']) to get them both optimized together. (How feasible that is remains to be seen.)

The current PoC is designed to allow both cross-file and cross-module optimizations (although this is not implemented yet), even when loading modules one by one. This is possible because if the cache for a dependency is invalidated, any modules that depend on it are also invalidated, so any cross- optimizations are safe.

Currently, require_modules(['symfony/cache/module.ini', 'symfony/cache/valkey/module.ini']) compiles and caches the modules separately.

I like Jordi's suggestion of init files per module. Rather than a hard-coded filename, though, could it just be another entry in the module.ini file? (Or whatever that evolves into, since I'm not a huge ini fan.)

Agreed. Maybe onload=init.php?

I really like the module chunks idea, which could also potentially help with minimizing the excess loading. Note that for some systems this would require changing their test namespace; Various conventions include putting Test in the namespace of test classes somewhere. I don't do that myself, but I've seen plenty of people that do. That code would need to be refactored to make it module-able. That could be a lot of busy work, but probably automatable, I'm guessing.

In theory, this can support chunks with the same namespace. The only limitation is that chunks must be in the same directory as the module.ini file. The following layout would be supported:

src/Foo/Bar.php (namespace Foo; class Bar)
Tests/Foo/BarTest.php (namespace Foo; class BarTest)

With the following configuration:

module=Foo
files=src/*.php
[tests]
files=Tests/*Test.php

However it may be challenging for autoloaders to know when to load the tests chunk: they can't do it solely based on namespace.

The way local visibility is described, would it work for properties and methods as well, or only classes/functions? We presumably want both. (Though as long as we're confident it could be done, visibility is a logical break point to split off to another RFC if we wanted to.)

I didn't though about properties and methods yet. I have to think about it, but it could probably work similarly, yes. Currently, visibility of class members is already implemented with name mangling, without knowing ahead of time if we are fetching a private or protected property.

If I read correctly, there's something about Enums that makes them... incompatible with modules? Hackliy supported? I don't quite follow the limitation there or what we can do about it.

Yes, there is some difficulty with Enums and classes that extend internal classes, because these can not be linked and cached in their linked state, currently (on Windows for internal classes, and all platforms for enums). Currently the PoC works around this in a very hacky way, and this has to be resolved. This doesn't seem impossible and this would benefit performance on Windows (maybe).

My original writeup never came up with a name for "a unit of compilation." It's still just "X". :-) Any suggestions on what that should be called, if it's even relevant?

At this point it's mostly a technical detail, and the real "unit of compilation" currently is individual files. However, the "unit of cache" will be the whole module tree (a module and its dependencies), although dependencies may be shared by multiple units.

I think we can remove this concept from the writeup for now, unless we need it to explain something.

I'm not clear if this part of the brainstorm is viable:

If the file declares a module that is already loaded AND is it not a child or sibling of the module.ini file that was used to load it, it is a fatal error. This allows the package to have module files that are not bulk-loaded (because they are rarely used), but prevents other packages from "sneaking" into a foreign module.

I don't know if this is fully handled by the chunks concept. Maybe it is. Thoughts?

There are two parts in this:

• Preventing sneaking symbols into a foreign module
• Deferring the load of some chunks of a module

Currently the first part is prevented entirely by disallowing require/include of a file with a module statement (unless we are loading that module), so it's not possible to add stuff in a module by loading just one file.

I'm not sure about the second part, however the three use-cases I have in mind would be handled:

• Having tests in the same local scope, but not loading them by default
• Loading parts of a module only when needed, for memory usage purposes
• Optional dependencies

All in all, the top line performance numbers (~10% performance boost already, and the potential for another ~10%) make this extremely interesting and valuable to pursue. The primary concern is ensuring we do not hurt any valid common workflows along the way.

Agreed

[arnaud-lb]

@derickr

Additionally, I read "Local symbols are mangled in the symbol table, so that looking up a local class yields nothing. As a result, normal class lookup is unchanged, and no overhead is added." — how are you proposing the mangled form looks like?

Anything like \0$module\0$class_name would work. zend_lookup_class_ex($class_name) would not find the class (as expected). I haven't though deeply about this yet, but the function may fallback to \0$module\0$class_name when $class_name was not found, with $module the module being compiled or the module in scope, if any. In non-module code the overhead would be zero when the class exists. In modules, static references to classes may be resolved at compile time.

[Crell]

The current PoC is designed to allow both cross-file and cross-module optimizations (although this is not implemented yet), even when loading modules one by one. This is possible because if the cache for a dependency is invalidated, any modules that depend on it are also invalidated, so any cross- optimizations are safe.

Currently, require_modules(['symfony/cache/module.ini', 'symfony/cache/valkey/module.ini']) compiles and caches the modules separately.

Hm. Does that mean the performance characteristics of these two samples would be the same?

require_modules(['symfony/cache/module.ini', 'symfony/cache/valkey/module.ini']);

// vs

require_modules(['symfony/cache/module.ini']);
require_modules(['symfony/cache/valkey/module.ini']);

If loading both modules in one command doesn't actually have an optimizer benefit, it's probably best to make it a single value rather than an array, for simplicity. (I expected that doing it together would give the optimizer more to work with, which is why I did it that way.)

[Crell]

Currently the first part is prevented entirely by disallowing require/include of a file with a module statement (unless we are loading that module), so it's not possible to add stuff in a module by loading just one file.

This would mean chunks are the only way to partially-load a module. I'm not sure about this, as it effectively means as soon as you are using modules, all current autoloaders break down. You have to go all-in on module-based loading, which is by design less fine-grained. Is disallowing requiring of a file in a module necessary, or just the easiest way to do it for the PoC?

[arnaud-lb]

Hm. Does that mean the performance characteristics of these two samples would be the same?

require_modules(['symfony/cache/module.ini', 'symfony/cache/valkey/module.ini']);

// vs

require_modules(['symfony/cache/module.ini']);
require_modules(['symfony/cache/valkey/module.ini']);

Yes

If loading both modules in one command doesn't actually have an optimizer benefit, it's probably best to make it a single value rather than an array, for simplicity. (I expected that doing it together would give the optimizer more to work with, which is why I did it that way.)

Yes this is what I understood, and I was not sure until now. I agree.

Currently the first part is prevented entirely by disallowing require/include of a file with a module statement (unless we are loading that module), so it's not possible to add stuff in a module by loading just one file.

This would mean chunks are the only way to partially-load a module. I'm not sure about this, as it effectively means as soon as you are using modules, all current autoloaders break down. You have to go all-in on module-based loading, which is by design less fine-grained. Is disallowing requiring of a file in a module necessary, or just the easiest way to do it for the PoC?

It was just the easiest way to do it for the PoC. It may be possible to allow loading of individual files, internally it would be very similar to how chunks would work.

This brings a few questions:

• Do we allow to require a module file if we haven't required the module?
  • If no, it means that the autoloader must be module-aware anyway
  • If yes, in theory we could use modules without calling require_module() at all. The only benefit of require_module() would be for non-PSR-4-style file organization, and the performance boost of not calling the autoloader for every class.
    We need to prevent loading either a single file or a whole module from a different directory later, so when requiring a single file, we have to register that we have partially loaded the module, from some root directory. We have to find the root directory by looking up the module.ini file.
• Do individual files loads behave like modules with regard to conditional declarations and side effects? (They are executed once when not cached, and subsequent requires will only load the classes and functions that were defined in the file.)