Document using the Shadow plugin as an alternative to Boot's fat jars when using Gradle

[btiernay]

I found the following is required in Gradle to get a fat jar working when cannot use the spring-boot-gradle-plugin plugin:

import com.github.jengelman.gradle.plugins.shadow.transformers.*

// Can't use Boot's nested jar packaging due to environment constrained classloader
apply plugin: 'com.github.johnrengelman.shadow'

shadowJar {
  zip64 true

  exclude 'META-INF/*.SF'
  exclude 'META-INF/*.DSA'
  exclude 'META-INF/*.RSA'

  // Required for Spring
  mergeServiceFiles()
  transform(AppendingTransformer) { resource = 'reference.conf'  }
  transform(AppendingTransformer) { resource = 'META-INF/spring.handlers'  }
  transform(AppendingTransformer) { resource = 'META-INF/spring.schemas'  }
  transform(AppendingTransformer) { resource = 'META-INF/spring.tooling'  }
  transform(PropertiesFileTransformer) { paths = ['META-INF/spring.factories' ] }

  // ...
}

The docs should strongly recommend the shadow / shade plugins for environmentally constrained classloaders¹ because resource transformation is a requirement. Without it, essential beans such as converters, property placeholder configurers, etc. go missing from Spring Boot's autoconfiguration (i.e. spring.factories). The result is very difficult to diagnose.

¹environmentally constrained classloaders: A system classloader not under your control.

[snicoll]

what do you mean not using the boot plugin? Are you creating a fat jar yourself?

[btiernay]

Yes, this is a requirement in certain environments such as Hadoop, Spark, Storm, etc. since they don't know how to load nested jars. Please see #1668 for more details on this topic.

[btiernay]

Here is the section in the docs im referring to
http://docs.spring.io/spring-boot/docs/current/reference/html/executable-jar.html#executable-jar-alternatives

[btiernay]

Just hit an issue in a new Spring Boot version. I needed to amend the above to have:

transform(PropertiesFileTransformer) { paths = ['META-INF/spring.factories' ] }

This seemed to have resolved my issue.

[unilama]

Final working example should be like that, strategy is required if you are using several components that contains spring.factories:

import com.github.jengelman.gradle.plugins.shadow.transformers.*
shadowJar {

    // Required for Spring
    mergeServiceFiles()
    append 'META-INF/spring.handlers'
    append 'META-INF/spring.schemas'
    append 'META-INF/spring.tooling'
    transform(PropertiesFileTransformer) {
        paths = ['META-INF/spring.factories' ]
        mergeStrategy = "append"
    }

}

I hope it will save someone's lot of time :)

[philwebb]

We're cleaning out the issue tracker and closing issues that we've not seen much demand to fix. Feel free to comment with additional justifications if you feel that this one should not have been closed.

[wilkinsona]

I think there might be some merit to doing something in this area. Spring Cloud Function (and FaaS in general) would benefit from an easy way to produce a shadowed jar. We really need something that's equivalent to the Maven Shade Plugin's configuration from the starter parent pom. That could take the form of documentation, or it could possibly take the form of some code in Boot's gradle plugin that sets up the various transformers.

[wilkinsona]

With the spring-boot-docs project, there's no easy way for us to test that the Shadow plugin configuration works as expected and I don't want to add an untested snippet. I will add a link to the Shadow plugin alongside the other alternatives though.

[raj-saxena]

@unilama thanks for your comment. I was looking for this as Google's Dataflow also doesn't work with Spring's Fat-jar.
For KotlinDSL, here's what I had to do to make it work -

plugins {
    id("com.github.johnrengelman.shadow") version "5.2.0"
}

...

tasks.withType<ShadowJar> {
    isZip64 = true
    // Required for Spring
    mergeServiceFiles()
    append("META-INF/spring.handlers")
    append("META-INF/spring.schemas")
    append("META-INF/spring.tooling")
    transform(PropertiesFileTransformer().apply {
        paths = listOf("META-INF/spring.factories")
        mergeStrategy = "append"
    })
}

[adrian-baker]

Another aspect of the Spring Gradle plugin that needs manually recreating is the exclusion of the developmentOnly from the resulting jar. By default shadowJar consumes the runtimeClasspath configuration, and the plugin applies runtimeClasspath.extendsFrom(developmentOnly), which means the examples above include developmentOnly dependencies in the resulting shadowedJar, which is likely not desired.

I've found this the simplest way to recreate that:

shadowJar {
  configurations = [project.configurations.productionRuntimeClasspath]

Hopefully my reading of #26686 is correct and the productionRuntimeClasspath configuration created by the plugin is considered "API":

[ShalabhR]

Has this issue been addressed in spring-boot 2.7.5 ? I was using the shadowJar task defined above with my app that used spring-boot v2.6.6 but after upgrading to 2.7.5 looks like it stopped working and I had to remove shadowJar. Can anyone confirm if there was an effort to fix in 2.7.5 ?

[wilkinsona]

@ShalabhR This was a documentation issue, adding to Boot's reference documentation a link to the Shadow plugin. As such, there was nothing to fix. If you have a build that works with 2.6.6 and fails with 2.7.5, you may have found a regression. If you open a new issue and provide a minimal sample that reproduces the problem, we'll take a look.

[Djaytan]

Because I struggled several hours solving my issue, here is what I have done with Spring Boot v3.0+ (but may work for v2.7+ as well... To be tested):

Spring Boot 2.7 introduced a new ‘META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports’ file for registering auto-configurations, while maintaining backwards compatibility with registration in ‘spring.factories’. With Spring Boot 3.0 release, support for registering auto-configurations in ‘spring.factories’ has been removed in favor of the imports file.

=> https://stackoverflow.com/q/74871072/10165346

This means that it is required to merge the org.springframework.boot.autoconfigure.AutoConfiguration.imports files as well. Starting from there, auto-configuration starts to work again!

This leads me to the following Gradle configuration for the shadowJar task (Kotlin DSL):

shadowJar {
  // Required for proper Spring Boot shading
  mergeServiceFiles()
  append("META-INF/spring.handlers")
  append("META-INF/spring.schemas")
  append("META-INF/spring.tooling")
  append("META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports")
  transform(PropertiesFileTransformer().apply {
    paths = listOf("META-INF/spring.factories")
    mergeStrategy = "append"
  })
}

Hope it will help.