OSGi support for AsciidoctorJ

[lefou]

As a follow up to #293 and #294, we came to the conclusion that it would be nice to have better support for running AsciidoctorJ in OSGi environments.

Currently, the bundle using the Asciidoctor API needs to set it up including knowing all the details (e.g. setup of JRuby and it's gems, classloading, ...). If more than one bundle want to do this, we end up having lots of Asciidoctor (and JRuby) instances in the same JVM, which is not so good. The proper OSGi way would be to register a service into the OSGi registry that covers the Asciidoctor API. Consumers do not need to setup Asciidoctor themselves but instead consume the service from the OSGi registry. Provided that such a service exists, classloading, versioning, bootstrapping and such things would no longer be an issue.

What's needed for this:

• Make the asciidoctor-core jar a proper OSGi bundle (MANIFEST.MF headers)
• Decide the service API, e.g. Asciidoctor interface or something else
• Write an OSGi Activator which registers the service into the service registry
• Write tests and documentation how to consume the service
• Think about a proper mechanism for extension, e.g. the Whiteboard Pattern comes to mind.

[lordofthejars]

I have never done anything in OSGi but I know the concepts. Do you know any simple library, example who uses this approach so we can see how it can be done?

[mojavelinux]

@lordofthejars What I recommend here is an OSGi Arquillian test. It's a perfect use case because we can isolate the environment and ensure the changes function properly without manual testing. It's also a good opportunity / excuse to play with OSGi container support in Arquillian.

[mojavelinux]

Of course, one example of AsciidoctorJ in use in OSGi is SlideshowFX.

[lefou]

@lordofthejars Do you need an example of how to register/use services in OSGi? I'm not sure I understand your request correctly.

@mojavelinux Yes, I already had a discussion wirth @twasyl about this topic and SlideshowFX might be a good testbed/client for this feature. Currently, it's use of AsciidoctorJ differs from my view of the strived target solution: SlideshowFX (SFX) uses one single bundle to provide it's own API which uses Asciidoctor internally. That bundle embeds AsciidoctorJ and all required dependencies including JRuby. That is, IMHO, not the preferred general solution we want to provide. It think we should:

• Use the official JRuby bundle (at least jruby-complete looks like a proper OSGi bundle)
  • JRuby provides some OSGi-related helper classes including OSGiScriptingContainer which we might want to use (although my experiment with it are not that successful)
• The AsciidoctorJ JAR should be enhanced to be a OSGi bundle
  • When activated: it should register the Asciidoctor interface as a service
  • When stopped: it should call Asciidoctor.shutdown() and deregister the service
• Consumers of the Asciidoctor service
  • must not embed any Asciidoctor or JRuby dependencies but declare proper Import-Package entries in their bundle headers
  • need to get the Asciidoctor service from the OSGi Service registry.
  • are not required to apply any special classloader tricks

I had some spare time yesterday and experimented with it. You can find a first start in my osgi branch: https://github.com/lefou/asciidoctorj/tree/osgi. But I haven't committed any of my later unsuccessful tests (yet).

Looks like the JRuby OSGi classes require some assumptions I weren't able to guess.

[lordofthejars]

I didn't know you are already working on it, so don't worry I do not need any example :) It was to start a PoC about what we mention on issue, but if you are already working then it is not necessary :)

Thanks.

[lefou]

In the next days I will not find the time to continue it. It was more some kind of over-optimistic trial to quick hack a solution. Unfortunately it wasn't as easy as I thought. :(

[lefou]

Of course, the part that makes asciidoctorj-core a bundle (lefou@35a9106, lefou@d01e2d5) should be already good enough. Feel free to cherry pick.

[lefou]

I opened PR #328 that makes the asciidoctorj-core JAR a proper OSGi bundle. This is only the first part of the work outlined in this ticket, but I thought I better get that (working) part in, as I don't have the time to continue my experiments for now.

[mojavelinux]

👍

[lefou]

PR #358 adds a OSGi bundle activator which registers a Asciidoctor instance into the OSGi registry.

@twasyl could you try it out in SlideshowFX? Instead of embedding the whole Asciidoctor (and JRuby) dependencies, it should be enough to have it as a separate activated bundle and acquire the Asciidoctor service from the service registry.

@mojavelinux Do you have any news about some arquillian-based infrastructure to test the bundle?

[robertpanzer]

I just tried again to do an arquillian integration test today with felix.
Unfortunately the Arquillian OSGI container adapter still requires a rather old version arquillian-core. (1.1.2.Final)
I'm somewhat reluctant to step back to this version. It would at least imply going back to junit 4.11 (Even though this would not cause any problems.)
But that's where I stopped with an OSGI integration test.

[robertpanzer]

I continued to test but stumbled over an error in jbosgi in https://github.com/jbosgi/jbosgi-metadata/blob/master/src/main/java/org/jboss/osgi/metadata/ManifestBuilder.java#L101 :
The Manifest from the Deployment is read and apparently reformatted to trim all lines to at most 256 bytes.
But if you check the source carefully the last part of the line can be up to 511 bytes and in out case I somehow hit 510 bytes.
Add one blank before and '\r\n' at the end the line reaches 513 bytes.
And this makes java.util.jar.Attributes.read() fail with an IOException with the message line too long when AbstractOSGIApplicationArchiveProcessor tries to get the bundle manifest.

I've just seen on Github that this issue was resolved .... TODAY!
jbosgi/jbosgi-metadata@1f80135

[robertpanzer]

If the Import-Package Attribute would be just a few characters longer or shorter this step should work. 😿

[lefou]

Hmm, the Import-Package is not very long, compared to some other OSGi bundles I've seen. What I don't understand, why do they trim the lines at all? OSGi requires a max lines length of 80 chars, longer lines continue on the next line with a leading space. The gradle build creates exactly such a manifest.

[lefou]

Could it be, that you execute under Windows (line.separater = '\r\n') but the Manifest files contains linux newlines ('\n')?

[robertpanzer]

Yes, I'm on Windows.
The point is that ManifestBuilder rebuilds the Manifest with line lengths of up to 511 + 1 (leading blank) + 1 or 2 for the newline.
Then it creates a new java.util.jar.Manifest from this repackaged stream.
And on my system I just hit the exact point where the error occurs.
Could work on Linux, but you will also stumble over this error if add one character ;-)
Will retry once the artifacts are available. (4.0.1.Final version of jbosgi-metadata is already tagged)
At the moment it would mean to rebuild a lot of modules myself. (like jbosgi-parent that's also not publicly available yet)

[lefou]

Ok, now I understand better. You could add some junk import as a workaround, e.g. "org.slf4j;resolution:=optional".

[robertpanzer]

I got to the point where JRuby does not find the Asciidoctor gem :)
I checked in my changes to https://github.com/robertpanzer/asciidoctorj/tree/OSGIIntegrationTest (Based on AsciidoctorJ 1.6.0)
Maybe you'd like to fetch this.

[robertpanzer]

I'll try to prepare a PR for your branch tomorrow, maybe better that way around.

[lefou]

I tried your branch and made some changes, but had no success yet. :-(

Where can I find some OSGi-specific docs for Arquillian?