Versioning of LSP4iJ and the impact of API evolution.

[mrglavas]

We're currently working on a new release of Liberty Tools for IntelliJ that would depend on LSP4iJ from the Marketplace (OpenLiberty/liberty-tools-intellij#608). We were hoping that we could declare a dependency on a specific version but recently learned from JetBrains that we have no control over that. Specifically they said:

1. The build process from Gradle must always use exactly the specified version from Gradle build file.
2. Whatever version of the plugin is installed in the IDE cannot be controlled from your plugin. You cannot specify a dependency on a minimum/maximum version of dependent plugins, see https://plugins.jetbrains.com/docs/intellij/plugin-dependencies.html#1-locating-plugin-id-and-preparing-sandbox

and also:

... you have no control from your plugin about what version of dependent plugin will be installed in the IDE. By default, IDE will choose latest compatible version from Marketplace.

I believe this does imply that an IntelliJ plug-in (installable on a specific version of IntelliJ) has to evolve in compatible ways for any other plug-in to reliably depend on it or adopters risk that their end users install a broken solution by default. To understand the scenario, we were working towards having a release of Liberty Tools for IntelliJ that would have been based off of LSP4iJ 0.0.1. We would have stabilized and tested our release around LSP4iJ 0.0.1. Immediately after our release a user would install our plug-in from the Marketplace with LSP4iJ 0.0.1. Some time later, LSP4iJ 0.0.2 is released. On that day a user installing our plug-in (same version) from the Marketplace would get LSP4iJ 0.0.2, a version that did not exist at the time of the Liberty Tools release and was not available for testing. I just became aware today of an API change that's targeted for LSP4iJ 0.0.2 (see StreamConnectionProvider - #348) that I would assume if released would be a breaking change and if a user installed our already released plug-in it would be broken. This concern applies equally to any adopters of LSP4iJ, including IntelliJ Quarkus Tools.

To help identify potential issues before a breaking change is released to the Marketplace, we're planning to do continuous integration testing (OpenLiberty/liberty-tools-intellij#815) with our releases of Liberty Tools for IntelliJ that use / depend on the LSP4iJ plug-in in the Marketplace. If we had a better idea of the planned release cadence for LSP4iJ that would also help us plan to test and provide feedback before a new release of LSP4iJ is made available. If at all possible when there is a need to restructure APIs, I wonder if that could be isolated to a specific range of IntelliJ releases so that the changes only become visible when the adopters explicitly move up to a newer level of IntelliJ (e.g. 2024.2) and do not break their existing releases which target earlier versions of IntelliJ.

I'm curious what others thoughts are on this. We want to adopt LSP4iJ but I am concerned about how it evolves from release to release and what impact that might have on the end users of the plug-ins that adopt LSP4iJ.

[angelozerr]

@mrglavas as I have contributed a lot to the Eclipse IDE, I am aware that breaking the API is a nightmare for adopters. We suffer from this issue with Jetbrains sometimes changing its API and marking some APIs that were not internal as internal.

But it's also a nightmare trying to support old APIs and the code can become horrible. As we are at the beginning of the project, I decided to make this API break because using VirtualFile is more powerful than using URI. (This PR fixes several issues with the root URI because we are using VirtualFile).

The change I am making is code that I had not written (coming from LSP4E), so it is difficult for me to say "no problem I will never break the API" since I do not master all the code of base.

But my goal is to avoid breaking the API. And if I break the API, I'll ping you like I did.

[mrglavas]

@angelozerr A lot of my perspective comes from being one of the maintainers of Apache Xerces for a couple of decades. I tend to be more conservative about change because I know first hand what kind of impact that can have, not just on the direct users of a common component but the entire ecosystem / software stack that relies on it. We often had to be creative in how we solved problems so that the vast number of applications using Xerces would continue to work with new versions. That can be difficult and at times also a nightmare but we tried to shield that from users as much as possible.

Thanks for the clarification on the recent PR. I do understand the technical reasons for the API change in this case. If we accept that the API will have breaking changes from time to time, I believe it would be good if there were some kind of policy for making such changes and guard rails in place to protect released plugins that depend on LSP4iJ from breaking.

It seems that when JetBrains wants to replace old APIs, they deprecate the old APIs and give plugin developers a reasonable amount of time to adopt the new API before removing the old one. While that's still a burden to plugin developers (since they eventually need to change their code) it doesn't just abruptly break their plugins. It seems these API changes are introduced on major releases (e.g. 2023.2, 2023.3, 2024.1, 2024.2, etc...) of IntelliJ and that their fix releases (e.g. 2023.2.1, 2023.2.2, etc...) are generally stable. Having a policy like this would be very helpful for adopters of LSP4iJ. Specifying a supported range of IntelliJ releases for LSP4iJ could be another way of shielding adopters from breaking changes. Also, the straightforward and simple change might be to break the API but were there other non-breaking options for accomplishing the same goal? When an update is needed to the API, it would be great if it were introduced in a way that avoids making a breaking change so that it does not impact adopters.

[fbricon]

@mrglavas as I replied elsewhere, we're very aware about the drawbacks of breaking API. Because LSP4IJ is still in its infancy, has only a handful of adopters and because we know you haven't yet released your plugin relying on LSP4IJ yet, we deemed it an acceptable risk to do it this time.
We would have had to jump through more convoluted hoops if you already released your plugin, so as not to break it.

[mrglavas]

@fbricon We understand this specific case and that it wouldn't have made sense to jump through hoops if no one was using that API yet. I'm only referencing it here as an example, but was intending to have this broader conversation anyway as we want to understand how this is all going to hold together in an ecosystem that allows dependencies to be upgraded beyond the level that the adopter initially released / tested with.

[TrevCraw]

@fbricon One of our goals for this discussion is to understand or work together to define what the process will be when a breaking change is made in the future (i.e. how adopters will be notified, how much time they will have to make the change, etc). We would also like to understand or define what guard rails will be in place to protect existing adopter plugins forced to use newer versions of LSP4IJ released to the marketplace with breaking changes (i.e. new LSP4IJ releases only support the latest version of IntelliJ, so existing plugins on older IntelliJ versions will remain unaffected and continue to use an older version of LSP4IJ).

[yeekangc]

From redhat-developer/intellij-quarkus#1342 (comment):

The team has been actively working to adopt a release of LSP4IJ formally. We are looking to do so in an upcoming release in the coming quarter as soon as possible. While we will manage the churn here, as per the discussion in this issue and else, it will be good to work out an approach that is more predictable and less disruptive when it comes to breaking changes. Let us continue the discussion and work out what's best for LSP4IJ and the community. Thank you.

[fbricon]

Aspirationally, we want to:

• provide support for latest jetbrains IDE - 1 year, i.e. for the last 3 major releases, i.e. if IJ 2024.2 is the latest version, we're trying to target 2023.2 as the minimum version. In terms of maintenance, supporting more JetBrains versions is an absolute PITA. They break something every 4 months.
• We don't have the resources to plan for a fixed release cadence, releases will occur "when ready/needed", but I don't think it'll be more often than once a quarter.
• We'll most likely move classes around in an actual API package over time, to make it obvious what is API and what is not.
• But we will deprecate LSP4IJ API for 1 year, before removing them entirely.
• it'd be nice to have an api-baseline check, like for Eclipse/Tycho projects, but I haven't looked into it for Gradle projects. If you know how to set it up, let us know
• We should adhere to semver as much as possible, where major releases can contain API breakages, and minor releases only get fixes. However, until we reach 1.0.0, you should expect 0.x.y to contain potential API changes, even though we definitely want to avoid breaking anyone as much as possible.

[fbricon]

FYI, LSP4IJ 0.0.2 has been published and is now waiting for JetBrains approval.

Next version will be 0.1.0, as it'll be easier to prepare hot fixes for it (0.1.x).

[fbricon]

let's make it 0.3.0, so my brains are less confused

[mrglavas]

Thanks for the update and anticipating my future question. 😄 We were curious how hot fixes would be handled.

[angelozerr]

Today we start to have a stable API and with LSP API which will be available in 0.7.0 (see PR #543) you will have the control to customize the LSP features (ex : ignore some CodeLens, etc) and disable some LSP features.

This PR will break some signature method from LanguageServiceAccessor but this class should be never used (see comments in the header class). If you need to access to the language server, we will need to provide a clean API but for the moment the only usecase that I see (and we need language server with our Qute LS) uses the language server stored in the LSP command context, so we don't need LanguageServiceAccessor.

I did the changed in intellij-quarkus yesterday, see https://github.com/redhat-developer/intellij-quarkus/pull/1391/files#diff-cbed23fcd78951a1a8362a9f5645b98dd8c4a404cce73bc88067d405a7ca1957R40

I close this discussion, but if you need to add more feedback, please reopen it.