Be able to register migrations from multiple folders

[jjanvier]

Hello Doctrine team,

At Akeneo, our migrations are stored in different folders (yes, we really need that). Would you be interested a PR that allows this feature ? Basically it seems that only the class Doctrine\DBAL\Migrations\Configuration\Configuration should be modified.

[NigelGreenway]

+1 for this.

I have been looking at the best way of doing this myself.

@jjanvier Out of interest, how do you separate yours? At the moment I have been looking at connection/release/versionxxxxxxxx.php

[chrisguitarguy]

@mikeSimonson I can help with this! Just need to know how you would want this to work? Just glob multiple directories for Version*.php? Or does there need to be separate namespaces?

I would think the namespace wouldn't matter. Just recursively search the directory, require_once every file, and look for all the classes from the directory filtering out the ones that don't extend AbstractMigration. Similar to what the doctrine persistence annotation driver does.

[mikeSimonson]

I would like to know why @jjanvier and @NigelGreenway need that feature to find the best way to do it.

If it's to have a folder to keep older migration, it make sense to me.
We wouldn't need to change the namespace and a simple recursive glob would work.

Is it what they are looking for ?

[chrisguitarguy]

I can think of a good reason for multiple directories and removing the namespace: integration with framework modules or bundles. Each one could hook it and register their own migrations.

[NigelGreenway]

The way I would like to use it is to separate them by feature, this way it would only be an extra level of directories.

Sorry I can do anything myself at the moment.

[mikeSimonson]

Separating by feature looks like a valid use case too but the multi integration does not. The migration are going to be different anyway, it makes more sense to have different folder and different config for different integration IMO.

[chrisguitarguy]

Would this require recursively searching a single directory? Or being able to register multiple directories in the config? Maybe both?

[mikeSimonson]

I think that we can probably get away just searching recursively the folder. The question is should we add an option to keep the old behavior ?

[chrisguitarguy]

I was going to pull out the "find and load files" stuff into a separate class, so if someone wanted to go back to the old behavior they could inject the old class into their configuration when creating the commands. Should be able to keep things compatible, so it should be transparent.

[stof]

Well, registering migrations from bundles would cause issues with the sorting of migrations: bundle migrations should be triggered based on the date at which the bundle was updated in the project bringing this migration, not based on the date the migration was added in the bundle. Otherwise it could break other existing migrations of the project (which would suddently run with the new bundle schema while they were written for the old one).
This is why registering migrations from bundles is not implemented in DoctrineMigrationsBundle: doing it is broken by design.

[chrisguitarguy]

@stof good call. Didn't think about that.

I'm too used to the migrations in Django, which are on a per-app basis.

[stof]

@chrisguitarguy and this is what DoctrineMigrationsBundle is doing too. But a Symfony project is an app (you can actually have multiple apps if you have multiple kernels by duplicating the app folder to create another app). Symfony bundles are not apps.

[chrisguitarguy]

Django apps are pretty much along the lines of what a Symfony bundle is: a reusable piece of functionality. Symfony has bundles and an application. Django has apps and sites.

Django batteries included-ness means everyone uses an ORM and the core table naming logic makes sure database conflicts don't happen between apps. Django can guarantee each app's migrations won't cause issues with other things.

To do migrations per bundle, Bundle's would need a way to "claim" tables and such to make sure nothing else messed with them. Schema object abstractions would probably have to be used and no more addSql. All that sounds like a big pain in the ass to implement and deal with, so I can understand the reasoning.

[stof]

@chrisguitarguy this would work if bundles were fully isolated from each other. But it is very common for bundles to have relations to tables of other bundles, or for the actual entity to be in the project itself, extending from a base class in the bundle, to allow adding extra fields.
And this does not change the fact that queries added at the project level may depend on fields coming from bundles.

And using the Schema object abstraction to write your migration is already possible (and causing a huge performance issue for anyone not using this feature btw, see #323 about that). But as I already explained several times (for instance in #79 (comment)), this has major drawbacks and is totally unsuited for production usage, because a production migration requires to migrate both the schema and the existing data. Running migrations on an empty database is not a real use case for production.

[goetas]

As of doctrine migrations 3.0 this is supported out of the box. I wrote a tutorial about it on https://www.goetas.com/blog/multi-namespace-migrations-with-doctrinemigrations-30/