Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a dependency-bundled jar #21

Closed
tresf opened this issue Apr 8, 2019 · 10 comments
Closed

Create a dependency-bundled jar #21

tresf opened this issue Apr 8, 2019 · 10 comments
Assignees
Labels
bug Something isn't working component:maven Changes to the java build process in-progress
Milestone

Comments

@tresf
Copy link

tresf commented Apr 8, 2019

Request to create an additional non-shaded, standalone jar for our downloads area.

  • This can have namespace clashes, classpath issues, these issues exist for other projects too (such as PDFBOX using BouncyCastle, etc)
  • This will result in two separate "fat" jars. The "uber" jar, which is maven-friendly, and the "release" jar which is available on our GitHub project page for download.

Original bug report:

Trying to run java-simple-serial-connector-2.8.1-SNAPSHOT.jar inside a project causes the following exception:

Caused by: java.lang.NoClassDefFoundError: org/scijava/nativelib/NativeLibraryUtil
	at jssc.SerialNativeInterface.<clinit>(SerialNativeInterface.java:81)
	at jssc.SerialPortList.<clinit>(SerialPortList.java:43)

I consider this a bug. Previous versions of JSSC did not have this issue.

@tresf tresf added the bug Something isn't working label Apr 8, 2019
@bmarwell
Copy link

bmarwell commented Apr 8, 2019

I see what you mean. I guess you're not using dependency resolution. If you were using ivy, maven or gradle in your project, dependencies would automatically be downloaded.

Simple fix:

Just drop the native-lib-loader jar on your classpath.
Also drop the slf4j-api.jar on your classpath, which is a dependency of "all" java projects. It is a de-facto standard in the Java world.
If you actually want to see log messages, also drop an implementation of your choice on your classpath.

More complicated fix:

Create a shaded jar, i.e. on a build move all classes from org.scijava.nativelib to something like jssc.shaded.org.scijava.nativelib.

The drawback is that those classes would be loaded multiple times in projects where there are multiple usages of the native-lib-loader artefact. Also this would mean: yet another profile.
Although technically possible, native-lib-loader has a simplified BSD license, and I don't know about this project. I have a commit at hand, but I need the license issues clarified first: Can we ship LGPL'ed code combined with Simplified BSD'd code?


I consider this a bug. Previous versions of JSSC did not have this issue.

I don’t agree, as dependencies should not be downloaded manually anyway. But I can see that this project has not had any dependencies in the past, so this might be new to users. So I'll leave the bug tag here for now, just for attention.

@tresf
Copy link
Author

tresf commented Apr 9, 2019

I don’t agree, as dependencies should not be downloaded manually anyway.

Then remove the dependency. The old version did this fine and this is a regression. Not all projects use maven. The title of the project is "Java Simple Serial Connector". It's no longer simple once you add 3rd party dependencies that break builds. Applying maven logic to all releases paradigms is wrong. Pragmatism doesn't win over usefulness.

@tresf
Copy link
Author

tresf commented Apr 9, 2019

Can we ship LGPL'ed code combined with Simplified BSD'd code?

Yes. It's a fact. BSD is compatible with just about anything. Not so much the other way around, but we're LGPL, so we're clear. BSD can be downgraded to LGPL (or even GPL for that matter) cleanly without any licensing implications.

those classes would be loaded multiple times in projects where there are multiple usages of the native-lib-loader artefact

Let's not confuse maven convention with binary releases. Take PDFBOX for example. They have slim versions and release versions. Release versions run with zero additional dependencies. So, say that you need PDFBOX in your boilerplate small project today, the fat release version works great.

Those concerned about having two bouncy-castles (for example), can safely use dependency management to fetch dependencies instead. I don't think it's right to break the lib over this.

@bmarwell
Copy link

bmarwell commented Apr 9, 2019

Some alternatives then, as I can agree to your argument:

Alternatives

  1. Either create a fork from native-lib-loader without that logging dependency and remove all the logging, maybe replacing it with Exceptions.
  2. Ship with a Noop-Logging-Implementation or JUL-implementation as a jar-with-shaded-dependencies. That means that the jssc.jar would contain classes like: io.github.javanative.jssc.shaded.org.scijava.nativelib.NativeLoader and same for slf4j. If this even works, because slf4j uses a loading mechanism.
  3. Copy stripped-down classes from the native-lib-loader to our project, shrink them as much as needed, and re-implement it as our own native library loader.
  4. Tell native-lib-loader to not use logging.

Pros / Cons

  • I don’t think (4) would be accepted. They intentionally switched to slf4j: https://github.com/scijava/native-lib-loader/pull/12/files.

  • I consider (2) out of the question, because it will make the jar file unnessecary big and bloats the pom.xml with yet another workaround. I don’t think it’s possible anyway, because slf4j looks for a specific class on the class path.

  • Solution (1) has the pro that we could accept patches from upstream, but requires further time investment.

  • Solution (3) elegantly replaces the provisional native library loader which was implemented by scream3r, but with a well-tested implementation. We won’t be able to integrate new features from upstream, but my guess is: We don’t need to anyway.
    Also, as soon as we have a multi-module maven project (some time in the future), we have a separate namespace in the same project and can maintain it much easier. Separate directory, but not separate project.
    Another plus: We can throw meaningful exceptions again. The upstream project just logs the error and does not load the library without letting the user know about it.
    That means we have better compatiblity with the last version provided by scream3r, which was your intention of opnening this issue in the first place.

Implementing (3)

That said, I'd suggest to create a new package in the java source directory: io.github.javanative.jssc.nativelib. This would contain the stripped down versions of the native lib loader.
Or I could implement our own lib loader if we don’t want to fiddle with two licenses.

@tresf
Copy link
Author

tresf commented Apr 9, 2019

Have you tried to run it without slf4j? I think it works. I've written no-op loggers before as well.

@tresf
Copy link
Author

tresf commented Apr 10, 2019

So I've talked to @bmhm about this on Slack and his concern is package "shading" through Maven will break the slf4j logger. If we can disable shading for release builds, I think that would fix it.

@tresf tresf changed the title NoClassDefFoundError: org/scijava/nativelib/NativeLibraryUtil Create a fat jar Apr 10, 2019
@tresf
Copy link
Author

tresf commented Apr 10, 2019

Subject changed to reflect goal of providing a "fat" jar that bundles stuff like slf4j but under it's existing namespace, not a shaded namespace.

tresf pushed a commit that referenced this issue Apr 10, 2019
* Change maven id and project name to just "jssc" at "io.github.java-native" namespace
* Adds two releasable jars (platform-specific jar and uber jar).  TODO: add non-shaded fat jar #21
* Allow cmake to parse version info from pom.xml
* Adds ability to skip native compilation steps including cmake and tests
* Leverages maven-assembly-plugin for better classifier to the default artifact
* Alphabetize, organize and remove hard-coding of dependencies/variables
* Adds Travis-CI for many platforms.  Note, some binaries are not yet usable, sorry!
* Adds Ubuntu 14.04 cross-compiled binaries to the source tree. TODO: Move to git-lfs
* Add LICENSE to pom.xml (#22)
@tresf tresf mentioned this issue Apr 10, 2019
@bmarwell
Copy link

Thanks for documenting. In the end we will have three main jars:

  1. ${project.groupId}:${project.artifactId}:jar::${project.version} (i.e. no classifier)
    The default jar anyone using a dependency management would expect. The pom would contain dependency information on (currently) native-lib-loader and that one on slf4j-api.
  2. ${project.groupId}:${project.artifactId}:jar:fat-jar-unshaded:${project.version}
    This artifact would contain all of the dependencies, even the transitive ones. At the time of writing, this is native-lib-loader and slf4j-api. Unshaded means, that their package names stays the way they are. If you used this dependency in, say, maven, it would clash if you also had other dependencies requiring either native-lib-loader or slf4j-api, because it would be twice on the class path.
    • ${project.groupId}:${project.artifactId}:jar:fat-jar-shaded:${project.version}
      I'll make this optional, as a shaded dependency is rarely needed nowadays. It's the same as the unshaded, but the dependencies go into a subpackage of io.com.github.javanative.jssc. That means, the classes are relocated and rewritten. This might be useful if you have other dependencies on your classpath also requiring (maybe another version) of either native-lib-loader or slf4j-api and want to use both at the same time.

The third option is optional, because with java modules on their way, I consider this the better option.

@bmarwell
Copy link

Let's close this as the maven branch has been merged and such a jar is generated on mvn -Ppackage.

@tresf
Copy link
Author

tresf commented Apr 14, 2019

No, it's not resolved. Dependencies are missing.

@tresf tresf reopened this Apr 14, 2019
@tresf tresf changed the title Create a fat jar Create a dependency-bundled jar Apr 14, 2019
@bmarwell bmarwell added this to the 3.0.0 milestone Apr 14, 2019
@bmarwell bmarwell added component:maven Changes to the java build process in-progress labels Apr 14, 2019
@tresf tresf closed this as completed in ddd6165 Apr 15, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working component:maven Changes to the java build process in-progress
Projects
None yet
Development

No branches or pull requests

2 participants