Call To Action: Secure the future maintenance of Xtext

[cdietrich]

Hello Xtext Community,

You might have observed that the number of people actively contributing to Xtext & Xtend has decreased over the recent years. People move on or shift focus. Of course this has implications: Fewer resources mean fewer new features, fewer bugfixes, and reduced community support. On the other hand, the outside world is turning faster: Four Eclipse releases a year, two Java releases, Gradle, Maven, etc. This means that the effort for maintenance and release engineering is increasing at the same time. Building four releases per year including a dozen milestones, adapting to changes in Eclipse Platform, JDT, keeping pace with new Gradle version, changes in new Java Versions etc - all these activities take their toll. Also with the changes in the team structure, fewer and fewer people have the knowledge about the internal workings and the implications of changes inside and outside of Xtext. In a nutshell: the future maintenance of Xtext is a risk.

As there are many people out there using Xtext in their tools and products, this risk has to be mitigated. We are thinking about what can be done to improve the situation. This basically boils down to two things. itemis AG provides a maintenance budget that can be extended by buying support contracts (contact us at xtext at itemis.de for more information or discuss the issue) and/or getting more people involved and contributing. In the past the development and maintenance of Xtext was always a one/two company thing and never a real community effort. So getting interested parties involved in the maintenance process is going to be challenging.

At EclipseCon in Ludwigsburg we talked to a number of interested parties about what we can do and came up with these ideas:

• Make the general maintenance tasks more transparent and document what needs to be done and what has to be done to be part of an Eclipse Release
• Have an open call (Hangout, …) to discuss the roadmap, what needs to be done and who can do what on a regularly basis (4-6 weeks)
• More automation of tasks
• Avoid duplicate tasks (mid-term, repo merge)
• Minimize the risk of things breaking (get rid of old xpand generator/dependency, don’t make extensive use of Xtend with its tight integration into JDT, ASM etc and also no longer encourage users to use Xtend for everything that Java can do as good or even better nowadays, Xtend is no longer the Java 17 of today)
• Think about which features are important and which one can be dropped to reduce the effort
• Think about the release cadence and being part of the Eclipse simultaneous release train.
• Find maintainers for subcomponents like Gradle support.
• Find a better chat-like communication platform than Gitter is atm

This is just a first idea. We invite you to bring in your own ideas into the discussion.

[ewillink]

Sadly you are leading the way on a problem that pervades at least all of the Eclipse Modeling projects. However you should be in a much stronger position so please lead the way. I'm pretty sure that you have many identified rich customers making good use of Xtext. These companies need to be encouraged / shamed into making financial / resource contributions to a support centre.

For me, Gradle is a complete waste of time; I have no idea what it even does. I suspect that it is what Maven ought to be (a not-XML language with 'completion' assist) but I lack the enthusiasm for yet another gratuitous change.

Again for me, Xtend has always been bad news; Java++----. I have consistently recommended writing 'Xtend' as a very thin Xtend layer on top of a thick Java layer. The thin Xtend layer is necessary for M2T String templates. Maybe string templates could reappear in a much simpler form.

[LorenzoBettini]

I think that first of all we should get rid of complex things in the overall infrastructure. For me there are two main points that should be dealt with immediately (of course, for what it can be done in the current bad and sad times):

• Single Git repository! As it used to be in the good ol' days: contributing was so much easier! Now creating an issue might involve provide PRs in several Git repositories (not to mention the temporary adjustment of the branch builds to consume new things proposed in other PRs). You will have to clone the entire Git repository even with things you're not currently interested in, but everything will be much easier.
• Build with Maven and with Maven only. It works! It's rock solid! Just stick with its dictatorial rigid lifecycle structure and it will work! Even Maven support in Eclipse is rock solid! It works! With Gradle in Eclipse having an error free workspace is the exception! Stop building with Gradle (sorry, but I still don't understand all the enthusiasm about Gradle). From what I understand many things break with new versions of Grade... the same does not happen with Maven. Read a Maven POM from any project around and you understand what it does and you can put your hands on it. The same cannot be said about a Gradle build (that's what I've heard at least). Of course, one might still provide gradle plugin support.

The two above points are meant to simplify things! IMHO they should be addressed immediately and should not be postponed.

It is my understanding that performing a release of Xtext is still a painful task. I think that's due to the wrong build infrastructure (too complex). Maybe it might also be due to the JIRO stuff (which I seem to understand did not simplify things... ;)

Concerning the Maven parts I can help. I was suggesting a user BOM some time ago, but then only a dev BOM was created... IMHO, a BOM is for users. I was told that it was too complex (IIRC, because of the complex Gradle/Maven stuff...). I've been working on an Xtext parent POM (currently for my own projects) that drastically simplifies the life of Xtext users and most of all the maintenance of the Maven structure from our side. I can contribute that if there's interest about that.

I already expressed my opinion about Xtend in particular how sad I was that it is not pushed anymore... I hope it's not going to abandoned completely. I'm not sure about that, but probably if also Xbase is abandoned I'll probably stop using Xtext.

[JanKoehnlein]

OK, here's a bit of a TypeFox perspective on this topic.

First, let me state that Xtext is a very stable and mature framework and overall in a very good shape. That's thanks to the maintenance efforts from the committers over the years and recently. But I also think that Xtext wouldn't immediately fall apart if these maintenance efforts were reduced, e.g. by leaving the release train.

The repository split allows each involved party to focus on the individual components they are interested in. And if nobody cares, it's completely OK to abandon a component, as we did for IntelliJ support. And if nobody should care about Xbase anymore, we can still go on with the other components.

For example, xtext-core offers runtime-only or LSP users a small and consistent repo that doesn't scare them away with complexity they don't need. I think that worked pretty well and it has helped to open Xtext to a much wider audience. In the meantime, we at TypeFox almost exclusively focus on xtext-core and language servers. We think it is the future of Xtext.

Being a plain Java repo, xtext-core does not need any Oomph setup, p2 build, target platform or other things that need a lot of time and special expertise for maintenance. Bundling it up again which components that require these would be very counterproductive.

I hope we find a better solution, maybe something with git submodules and a simplified build.

Unfortunately, we don't have a good alternative for Xtend yet. I second that we shouldn't use it for normal code anymore, but for code generation it's still by far the best we have. I tried Kotlin, but like Xtend only works well in Eclipse, Kotlin only really works in IDEA. Maybe somebody knows a reasonably good alternative?

[cdietrich]

yes. but xtext-core is build with gradle. and keeping gradle and the gradle plugin working is work nobody is willing to do

[ewillink]

xtext-core is a complete red herring. There is useful Xtext and there is archive Xtext, In so far as useful Xtext is not a single entity, it is a major PITA.

It seems to me that the unfortunate decision to target IntelliJ caused a lot of bad decisions that have irritated at least some Eclipse users and benefited zero IntelliJ users. GitHub Issues rather than / as well as Bugzilla. Gradle rather than Tycho. GitHub rather than EF hosting ...

Five years ago Xtext was also a modular pain, but one learnt to install MWE2, then a bit of Xpand2 then Xtend then Xtext then MWE2-lang. The composite repos have helped here. Xtext code could be checked out from the one GIT. Since GIT fragmented, I'm just not interested in Xtext's multi-GIT its just too damn hard. Fortunately the good old fashioned Import Plugins and Fragments helped with my last problem.

It would be really good if Xtext went back to one EF GIT, possibly folding in MWE2, Xpand2, Xtend so it is all in the one place. Then users could just checkout the non-archive stuff and it would just auto-build. No need for any OOMPH which never seems to work for my use cases. Reverting to a straightforward Tycho build would solve many other problems.

[JanKoehnlein]

We are not married to Gradle, and we don't plan to maintain the Gradle plug-in either. I propose we migrate the xtext-core build back to Maven. Personally, I'd prefer a plain Maven (non-Tycho) build, because last time I used Tycho (that's a while ago already), it took ages to build because it was scraping a gazilion p2 update sites first and sometimes even failed. If that's no longer the case, I'd be OK with Tycho, too.

TypeFox could do the build migration to keep xtext-core separated. If it makes your life easier, feel free to merge the remaining components and simplify the build. I just wanted to point out that it may be a good moment to decide which of these you want to maintain in the long run before merging them into a giant blob again.

[cdietrich]

i dont think its a good idea to just consider xtext.core and ignoring the rest. as downstream components still depend on xtext-core we need at least an approch that still allows to maintain xtext-eclipse without too many hazards.

[JanKoehnlein]

What are the concrete issues/hazards you see with the separation of xtext-core? Maybe we find solutions that do not involve having to merge all repos.

[cdietrich]

i just want to make sure the eclipse part can be still maintained without having to do extra extra work which would make the situation worse than it is today

[JanKoehnlein]

Please elaborate why a Maven build (which is not the Gradle build nobody seems to be eager to maintain) would make the situation worse than it is today and what your main pain points are with the xtext-core separation. I think I provided a lot of reasons for the xtext-core separation and I am still not clear what you want.

[cdietrich]

i dont know it. i just want to make sure the development works as seamless as it works right now.
we know with gradle this work. so we should make sure it works with the new solution too.

if we just have a look at xtext-core and ignore or dont care about the rest this "make sure" is not done

[hannesN]

Here are my thoughts to the current state of Xtext:

There are way to many repositories and build-systems in use right now. I understand that a split between xtext-core and (let's call it) xtext-eclipse might be useful, but 7 repositories come on.
If size is a concern I would suggest archiving all git repositories and merge them back to one or two and get rid of the historical files. Nobody needs Xtext 1 or 2.0 in the repository.

I also find it a bit concerning that the maintenance is more concerned with the build process than with the original code. Especially looking from the view of a complete newcomer, Xtext is hell. What we need is a well documented code, generators which are not generating code which throughs warnings or generate comments with URLs which lead to nowhere. This is just embarrassing!
I understand the manpower is low. Issues need to be prioritized, but especially the Formatter2 API needs to be finalized.

And here comes the next problem. Because of almost no code is commented or explained, part like the formatter and serializer can't be maintained by anyone but the original programmer. If they leave the team, well we stuck with what we have.

My suggestion: Maintain 2.x but start with a Xtext 3.0. I wouldn't suggest a complete rewrite, however we need to do the following things:

• simplify and document(!) the code, get rid of old APIs and especially get rid of the interface extension pattern and the many instanceof checks if a class implements the newest extension.
• modularize the UI try to get rid of E3 dependencies or at least but them in an additional plugin
• use ONE build system for all modules/bundles and regarding to the eclipse way this should be maven

If the code is easier to understand, the strong dependency to the eclipse IDE is gone, maybe we can find more people interested not only in usage but in contributing to Xtext.

[LorenzoBettini]

We are not married to Gradle, and we don't plan to maintain the Gradle plug-in either. I propose we migrate the xtext-core build back to Maven. Personally, I'd prefer a plain Maven (non-Tycho) build, because last time I used Tycho (that's a while ago already), it took ages to build because it was scraping a gazilion p2 update sites first and sometimes even failed. If that's no longer the case, I'd be OK with Tycho, too.

@JanKoehnlein Tycho is much much better nowadays and it's not the old nightmare anymore. Having a pure Maven build for xtext-core might lead to some additional effort to then turn its JARs into bundles. With that respect, I'd still propose to stick with a Maven/Tycho build for xtext-core itself. It would then be straightforward to create both a p2 site and deploy its bundles as Maven artifacts to Maven central.

[JanKoehnlein]

@LorenzoBettini Thanks, it looks like I am fine with Tycho then.

For all: The main goal of this issue should be to figure out how we can reduce maintenance efforts. So please try to propose new work items only if you're committed to do the work and if that really reduces the maintenance effort in short-term. Please stay constructive by saving your personal frustration and global visions for a better opportunity.

I explained the TypeFox perspective: We want to keep xtext-core as this component is strategically most valuable to us in its current separation. I offered to do the migration of the build which would allow to drop the xtext-gradle-plugin.

Others my have interest in other components. They should please stand up and say what the particular maintenance problems with these are. Then we can try to find a good solution for everybody who's in.

In the end there will be some components nobody wants to maintain. And that's OK. itemis rang the alarm bell loudly (my ears are still ringing) and if no sponsor steps up, we will archive those. As I wrote before, this doesn't mean they are immediately lost. But they will fade away if nobody kicks in. There is no obligation to maintain everything for free forever.

Focussing on the essential components is the only way I see that allows to keep the project maintainable and fit for the future. I am convinced that prematurely merging all repos forces an all-or-nothing situation that will kill the project.

[cdietrich]

just to make it clear: the number resources doing such a thing like: make sure we work with the new ASM 8.0 or 8.1 that pops up on eclipse orbit or work with a new lsp4j version is somewhere between 1 and 2. so we are not talking about developing new features or do bigger bugfixes. it is really about the pure maintenance.
exaggerated: if i would not be able to work for lets say 4-6 weeks we would basically have to drop out from simrel immediatly

[ewillink]

Indeed Tycho is very useable, not perfect but one gets used to it. It certainly has many features to accommodate Eclipse. The M2E integration steadily improves. I no longer have fond memories that make me wish to revert to Buckminster.

If you do a full Xtext build, you can spit out an xtext-core sub-repo. In the past this was pretty easy with Buckminster. I never bothered to migrate the functionality to Tycho since the days when memory strapped users wanted a tiny OCL runtime are long gone. They can pull in only a few plugins if that's what they want all by themselves.

[spoenemann]

IMO going back to a monolithic source repository would not solve the original problem (too few contributors for maintenance work), but rather make it worse by scaring away everybody (huge code base, several technologies involved, >1h build times). The real solution is to further decouple the different components of Xtext.

My proposals:

• Decouple the release cycles of lib, core, eclipse etc.
• Drop out of the Eclipse release train.
• Reduce / eliminate use of Xtend in xtext-core (use the generated Java instead).
• Find sponsors for specific tasks (e.g. ASM update) instead of a general "please sponsor Xtext" call.

[ewillink]

What do you mean by build time?

If you are checking Java out of GIT, the auto-build time should be much less than 5 minutes. If not then maybe Xtend is too slow and needs eliminating.

If you mean 'Tycho' build and test, then yes it will take a long time, but a good tool has many tests and that takes time. If it's a real problem you can modularize your tests so that you run only relevant subsets by launching e.g. the xtext-core subset.

[ewillink]

If Xtext drops out of SimRel, it will be difficult for OCL, QVTd and presumably MWE2 and Xcore to stay on the train. If OCL goes then Papyrus, QVTo, ... may be in difficulty.

[tivervac]

What do you mean by build time?

The Xtend compiler is mostly single threaded and very slow, that is where the build times come from.

A lot of work is being done as we speak to reduce the amount of Xtend code.

[LorenzoBettini]

@tivervac if you're talking about Xtend compiler being slow during the build of Xtext base code itself, then the Xtend compiler might be skipped completely during the build since the Xtext Git repositories contain also the generated Java code. It could still be enabled only before releasing since we need the Java trace files for debugging purposes.

[maxweissboeck]

Well, we started a migration from a Visio & Oracle Forms kind of DSL to Xtext mid last year (we are in beta test now) so this could be the kind of message to make us nervous.

So some Questions from me

• Where would be the starting point / information if thinking about contributing to Xtext?
• Do you have any ideas how many active projects with Xtext there are out in the world?
• And why is there no response from the users (like me) to your message so far?

Best regards and thumbs up for Xtext!

[cdietrich]

a first starting point is here: https://github.com/eclipse/xtext/blob/master/CONTRIBUTING.md

as mentioned previously having the code split over 8 repos does not make it easy to contribute so this must be addressed in some form.

one problem with Xtext is that we have no idea how many active projects are out there and how to reach them. thus i dont know how many users of Xtext we I have already reached. Thus i also cannot tell why there are no other user give any signal at all here. But i still think that Xtext is used in a significant number of projects

[miklossy]

Hi Maximilian!

Where would be the starting point / information if thinking about contributing to Xtext?

https://github.com/eclipse/xtext/blob/master/CONTRIBUTING.md

Do you have any ideas how many active projects with Xtext there are out in the world?

https://www.eclipse.org/Xtext/community.html

Hope that helps,
Tamás

[cdietrich]

@miklossy this show public projects only.

[miklossy]

@miklossy this show public projects only.

True.

[maxweissboeck]

one problem with Xtext is that we have no idea how many active projects are out there and how to reach them. thus i dont know how many users of Xtext we I have already reached. Thus i also cannot tell why there are no other user give any signal at all here. But i still think that Xtext is used in a significant number of projects

Regarding to https://www.eclipse.org/forums/index.php/f/27/ current posts are viewed by some 100 people up to some 1000 people

So just speculating, there should be some 1000 Xtext projects outside there, or would people still read posts even without using Xtext actively?

[ewillink]

For the Eclipse Forum, and presumably GitHub, it should be possible for a committer to count the number of subscribed email addresses. Much more reliable than the views count. Once contributing I have probably viewed this thread ten times, while other threads I have been happy to read in Thunderbird and so probably not viewed at all.

[cdietrich]

@RBirdwatcher the problem stated is still true and the situation gets worse and worse.
so do you have any proposals the get the problem solved?

[mundacho]

Hello,

I've been closely following the Xtext development since the beginning of the year and I'm aware of the need for someone to do the maintenance. I just saw a message that the problem continues, and I think there is a solution: Open Collective.

Open Collective is basically a website to help fund open source projects. People can contribute money, and then it is paid to the people who actually maintain the software. I've seen it work with a web framework I used that was in exactly the same situation as Xtext, Playframework. Basically, the company that created it (Lightbend) stopped maintaining it, and they decided to setup an Open Collective account for the project so that the community can pay for its maintenance.

Now, companies and individuals using it are donating to the open collective and payments are being made to a contributor that maintains it. Here is the website of that project http://opencollective.com/playframework . It seems have worked for them and the situation seems to be improving. At least they are now having regular releases.

Maybe Itemis or the Eclipse foundation (I don't know who has to do it) can do this for Xtext, and leave it in the hands of the community. We just need someone willing to maintain it, maybe @cdietrich if he wants and can (since he is the one who does it now). I would happily do it, I've been trying to contribute to the project for a while, but in my free time there is a limit to what can be contributed to a project as large as Xtext. Maybe there is also someone else with a lot of experience who can volunteer.

What do you think?

[ewillink]

Open Collective seems interesting. A step towards what's needed, provided those with money also have consciences.

The Collective must be Xtext (or OCL or UML2 or ...) to ensure that the money goes to the project. A bitter experience with occurred when a generous donor did make a significant donation to 'rescue' QVTo, the donation was swallowed by Eclipse and no QVTo committer saw any specific benefit other than the generic EF support for GIT / Jenkins / ...

I gather that the EF is experiencing almost record membership so I'm unclear why there is no EF trickle down funding, at least for significant and under threat projects such as Xtext.

[cdietrich]

@ewillink this is basically the same information we got from the foundation when we evaluated the possibilities years ago. They don’t want projects to be directly sponsored

[ewillink]

I don't follow 'same'. The EF has no problem with direct project sponsorship whenever a committer is a paid employee of a company so I see no reason why a project should not distribute its income to its members. If necessary set up a nominal company whose directors are the committers and whose bank account receives the donations and distributes to directors / expenses. I just caution against letting the EF get its hands on donations since my experience is that our money vanished.

[lppedd]

I've had a read of all the discussion here.
I'll be short in explaining why people don't contribute to Eclipse projects involving PDE (but even to some that don't), with just three points.

1. p2. Plugins. Features. a.k.a complexity. I work with PDE (and RCP) since ages and it's always been a pain.
   Each time I work with plain Java project my stress level gets back to zero.
   Also, this technology is becoming niche, if it isn't already.

2. Eclipse IDE is pretty much mandatory. No Eclipse, no p2, no party.
   Now I bet some of you will say "We've got Tycho, Maven! Yay!". Well unfortunately Tycho doesn't play well with any other IDE, as it's pretty much another version of Maven. And I know it to the extent I've built my own IntelliJ IDEA version to support it.
   You can build through the CLI, but importing a project and getting the IDE to know what the heck it's editing is impossible right now.
   New engineers want VS Code, IntelliJ, the cool stuff, and they're willing to reject job offers to avoid Eclipse, that says a lot.

3. Setting up an Eclipse development environment is a mess, or an hit and miss at best.

The strategy for Eclipse projects should be to extract all the core functionalities to plain Java projects (Maven, Gradle, whatever is cross-IDE), and inject those dependencies in p2 offerings when needed. This allows core contributors to focus on plain Java. Eclipse lovers to keep their Eclipse integrations up to date. Same for whatever other community-driven IDE integration.

DevEx should be the primary focus, always.
And sometimes DevEx means dropping stuff that causes pain on every code change just for the sake of backward compatibility.

An additional note: IntelliJ IDEA Ultimate supports LSP now.

[Michka1111]

I've not read all these 77 comments written in ~3 years, meaning ~2 per month, this is not so much.
I began with Xtext a month or so ago, let me say that I found the framework Xtext (and Xtend) marvelous, and am very enthusiastic to start my DSL with.
I do not agree with so many thoughts against Eclipse. The integration of Xtext within Eclipse contributes to its richness. The Eclipse framework appears to be useful and stable, and it works. Just I would like to know what is happening in the background a little more.
I use the EDSL 2023-12 Eclipse Package and review the examples, the "7 languages in 7 days" along with those of the "Implementing a DSL with XText and Xtend - 2nd Ed." and I'm able to do the projects as explained. So, in my opinion, Eclipse is not the reason why contributions are decreasing.
The fact is that now I've done the samples, I want to start my DSL. And at this point, the pain begins. I'm stuck.
I know how to write the grammar, I know implement tests, validators, compilers, interpreters, write Xtend code, etc.
However, I do not know how to use the projects .ui, .ide, and .sdk for my DSL.
The 7 language examples show many snippets of code without referencing which sub-project and which file to put the code in.
It seems that the goal is just to write code, and edit source DSL, but not compile, and not interpret, and not integrate into my application. I could write a grammar to drive an ECS System, i.e. typed entities, components, and systems with methods to call, (even Copilot does it), but after? I want to run the commands of my source DSL, how to do this? I'd like to write a grammar to collect data to load into the components, I'd like to call the methods I defined in my DSL for the ECS's systems, how to call them from my DSL? which classes to code, in which file, in which subproject to put them? Which are and where to put the "inject extensions"?
It is said that Xbase comes with a compiler and interpreter, where are they? How do I use them? How to run all of these?
What are the details behind the scene of a Helper? I did not see any explanation about all that.
All these with no answer, in my opinion, let the enthusiasm decrease.
After 40 years of computing, all the technologies that grow up are those that answer these kinds of questions.
Xtext documentation has links to Javadoc pages which are just unreadable and bring nothing.
I'm sure not being alone in this situation, the decrease in the contributions seems to prove it.
I'm dressing up a mind map to solve this problem, but it is very time-consuming exploring the examples and the subprojects, understanding the underlying technologies, and then completing the map, but…
Maybe there should be more pictures that say more than long speeches when needed.
Hope this scream of heart will not rest in silence!
I'm very serious: it's just about laughing, this is driving life!

[miklossy]

@Michka1111 Thanks for sharing your thoughts with us!

Xtext documentation has links to Javadoc pages which are just unreadable and bring nothing.

Could you please enumerate on which places this exactly happens, e.g in a new GitHub issue?

[Michka1111]

From the Xtext doc:
at random:

• Xtext documentation / Page Language Implementation / §Code Generation
• "In this section we’re going to show how to implement a code generator for an Xtext-based language."
• "If you go with the default MWE workflow for your language and you haven’t used Xbase, you are provided with a callback stub that implements [IGenerator2] by extending the [AbstractGenerator] base class."
• Then go to the two links: it's nearly empty, and the comments are just trivial because I do not what to do. What is the development process, how do I implement such an interface, in which subproject in which already created package (by the project assistant, this is obvious for me) or do I create a package?
• Do you really think that after reading this section I am able to "implement a code generator for an Xtext-based language", as stated? And by the way, I do not find the same kind of section for Interpreter, again as stated.

From the 7 languages:

• From 05_guice.md / § Translation to Java
• "A module is mapped to a single java class. The 'mixin' modules are not translated to Java inheritance but to a delegation approach. Here is how a simple module declaration with a single mixed-in module is translated."
• "DSL:" / OtherModule {} / MyModule mixin OtherModule {}
• "Java:" / [skipped]
• "As you can see, the language simply leverages the nice equals/hashCode implementation of Guice's [Key] to implement the override semantics of mixed-in modules.
• There you go to the link [Key], this is the Javadoc page I did mention, I do not understand how I use this for my DSL:
• "Guice uses Key objects to identify a dependency that can be resolved by the Guice [Injector]. A Guice key consists of an injection type and an optional annotation."
• Again: what should I really do with these empty kinds of stuff to implement this in my DSL?

Just to say that the more I go through the doc, no didacts, no useful comments on what I have to do, the more I am in the darkness.
Clearly, I know software development (C, C++, Smalltalk, SQL, ksh, lex-awk-yacc, I want to learn Xtext, Java, Injection, Keys, all that, but I want do this through my DSL implementation, not spend hours off Xtext, learn, do other examples opposite of my DSL, and then go back to Xtext and do, even not sure after learned away from Xtext I'll know how to apply.

Even the links to Xtext files in the GitHub site are difficult to read because absolutely nothing is telling me how to apply to my DSL. How the subprojects are interrelated, and how to implement these interrelations, so answering to my fundamental questions.

[cdietrich]

the main painpoint here: Xtext is so flexible there is 10_000_000_000 ways of doing things....

[Michka1111]

Sorry, but no, there is only 1 way I'll implement my DSL, and then run it, and please help me to choose this one way among the 10^10 ways…

[cdietrich]

again: depending on the usecase each of these ways will be the correct for you.
for a general walktrough i can recommend you lorenzo bettinis book.

btw did you follow "the tutorial".
all the rest of docs is "reference docs"

[opcoach]

@Michka1111 if you need to use your xtend generators in an Eclipse UI, it is possible to call them in E4 commands using model fragments.. I can give you some tips for that... And this is exactly the training I am starting today in Paris ! We have xtend generators on ecore or on a business model and we connect them using ui selection and E4 commands...

[tripleo1]

My two cents:

Javadoc in itself is hard to read and mostly not useful, no matter what the project. This is based on the stylesheet(s) and the fact that there is so much more to know.

A lot of documentation is out of date, like I tried the sample projects from dsl-2023-06 or 09, and at least one did not run with the workbench/space/whatever it is distributed with.

IntelliJ EDU has kind of the right idea ;), and I'm not so sure that eclipse didn't influence it.

[Michka1111]

Hello, I've read the head post of this thread, because many questions pop up to decide if I go further with Xtext or if I look for another framework.
Why abandon Xtend/Xbase?
In what "Xtend is no longer the Java 17 of today"?
Where is the discussion for this where I could find more information?
Because all the literature & examples on Xtext are related to Xtend.

[Michka1111]

A little testimony:
I've looked at Langium to see if I've found answers to my fundamental questions. Langium appears based on Xtext.
Langium's grammar seems similar to Xtext's.
But there is ONE different keyword, a keyword used daily, hourly, minutely, secondly, everywhere in the two grammar and documentation, respectively.
In Xtext, this keyword is 'RETURNS'.
In Langium, it becomes "INFERS"!
To my knowledge, it made "bingo"!
All went enlightened!
Then, the schema of the Xtext parsing process popped up instantly under my pencil!
This impacts deeply my understanding of Xtext. And comforts me to go further with Xtext.
So, now I'm aware of the future maintenance of Xtext, despite I'm not (yet) a maintainer, but because I'm a user.
I think a major question is "To whom is made Xtext for?"
To its "citizen" domain expert users?
Or to its Java-advanced users who speak to the domain experts?
Or only to its maintainers?
Is the future maintenance wanted to simplify the hard work of maintainers or to simplify the usability for citizen domain experts?
In my experience, it'll be both.

[ewillink]

When it came out, Xtend could be viewed as a Java++ because Java had stalled. Now that Java is moving (too) fast with many developers, there is no realistic way that Xtend can remain ahead of Java. Also Java is stealing many of Xtend's good ideas, even string templates albeit with a very inferior prefix/suffix sequence.

[Michka1111]

Ok for Xtend which had its glory hour, and now tends to be history.
So, about this, what users have to do is not change the default value of the "preferXtendStub" in the .mwe2 workflow, right?
Or are there other hidden things to do?
How to migrate or refactor the existing tutorial samples and practices? (as for Newbees, this is just what they have to learn Xtext)
Or are there some incompatibilities in newer Java editions when still using Xtend, for instance in the java that is generated from Xtend?
What are the best practices of newer Java editions that apply to Xtext use cases?

What about Xbase? As Xbase leverages Xtext for Xtext users, I see a real added value.
For instance, What about XBlockExpression or JvmTypeBuilder when remaining in pure Java?
Is Xbase maintained relative to new features of Java?
How to refactor?

For a user, this is not that simple, or do I miss something?

On this point, my opinion is that Xtext is evolving for maintainers rather than for (new) users, leading to an actual decrease in contributions.

[cdietrich]

On this point, my opinion is that Xtext is evolving for maintainers rather than for (new) users, leading to an actual decrease in contributions.

the capacity of the project is below the barrier "evolve for new users" 2 years+ before this post was created.
specially about the docs. when they are insufficient. making it better would be something for a user to feed back their learnings

[ewillink]

IMHO Xbase is very important, and from my OCL-developer perspective I am very jealous. It is really hard for users to add OCL as an expression language for their grammars. Adding Xbase is fairly easy. Supporting 100% of Java is not necessary and actually undesirable. How many user grammars want the most abstruse Java functionalities and all their reserved word/punctuation corrolaries? Xbase should be a basic friendly familiar language. If someone really wants full Java they should develop an XJava extension.

[Naum-Tomov]

I think it may be relevant in this discussion to mention that Xtext remains relevant in academia. In terms of citations, there doesn't seem to be any significant decline, with 2023 even beating 2022 in terms of citations.

Source: scholar.google.com

[miklossy]

Nice to hear that @Naum-Tomov ! Thanks for sharing this information with us!

[ansgarradermacher]

Dear all,
I just want to let you know that the code generators in the Papyrus extension SW Designer are all based on xtend, i.e. we would need to find a migration path if xtend is no longer part of the Eclipse train (while the text templates in Java 21 are promising, they do not support control structures and the nesting of calls with proper indentation. I have done some experiments with lambda expressions to achieve loops within a text template, but it is too early to say if that would enable a smooth migration).
We use xtext editors in Papyrus core, currently in combination with xtend. The dependency can partly be solved by re-generation, but there is some custom code in xtend (relatively easy to migrate). The tooling of GMF diagrams is affected as well, but less problematic as upcoming Papyrus versions will switch to Sirius-based diagrams.

[LorenzoBettini]

@ansgarradermacher Currently, Xtend is still part of the Eclipse train, and I personally have contributed and am contributing a lot to it. You can use Java 10 text blocks and Java 21 templates ONLY for very simple cases; in that respect, nothing in Java beats Xtend multi-line strings ;)

[ansgarradermacher]

Thanks @LorenzoBettini, good to know! (of course, not a guarantee for a longer term perspective)

[LorenzoBettini]

@ansgarradermacher WOW! Because there are things that are guaranteed forever! ;)
Sorry for the sarcasm, and don't take it personally.
But besides contributing almost daily I cannot provide any further guarantees.

[maxweissboeck]

Off Topic but worth to mention. Lorenzo, I really appreciate your book, it helped me a lot to build a really cool Xtext application as a replacement for a "monstrous" Oracle Forms/Visio application. I can only recommend the book to anyone who is not yet an Xtext expert. Best regards Max

…

Am 10.04.2024 um 10:11 schrieb Lorenzo Bettini ***@***.***>: @ansgarradermacher <https://github.com/ansgarradermacher> WOW! Because there are things that are guaranteed forever! ;) Sorry for the sarcasm, and don't take it personally. But besides contributing almost daily I cannot provide any further guarantees. — Reply to this email directly, view it on GitHub <#1721 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIRDSJ4C3QQYJCRXQOM6YNTY4TX25AVCNFSM4LWQOK72U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBUGY4DINZSGUYQ>. You are receiving this because you were mentioned.