Packaging issue for adapter-aws

[gotson]

Hi,

i know this is not exactly a spring-cloud-function issue, but definitely related.
I am trying to package a project using the aws adapter, following the sample provided.

For my first project it is working fine, but for the second one i have an issue with the generated jar.
The spring.factories file is missing the EnableAutoConfiguration.

Working project spring.factories content:

org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
org.springframework.cloud.function.context.config.ContextFunctionCatalogAutoConfiguration

Non-working project spring.factories content:

org.springframework.data.web.config.SpringDataJacksonModules=org.springframework.data.web.config.SpringDataJacksonConfiguration

I managed to narrow down the problem to one of the dependency used in the non-working project (by removing it, the generated -aws.jar has the correct spring.factories file), which is: com.github.spring-data-dynamodb:spring-data-dynamodb:4.5.5

I tried to dig into the documentation of the spring-boot-thin-launcher, but i couldn't get my head around how it's working exactly, and what exactly is the problem with this dependency in the first place.

My guess is that some transitive dependency of spring-data-dynamodb is including some part of the spring core, which is messing up the thin jar layout.

I am using Gradle, with a build.gradle that is greatly similar to the one in the aws sample.

Any help is appreciated !

[gotson]

I tried to reproduce from the aws sample, but couldn't.
However, i managed to fix the problem on my end by adding the following directly inside resources/META-INF/spring.factories:

org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
org.springframework.cloud.function.context.config.ContextFunctionCatalogAutoConfiguration

It gets merged into the final shaded -aws.jar.

I'm still not sure why it's not working out of the box though.

[dsyer]

Seems like probably a gradle issue. You don't need to build a thin jar for any of the features of the sample to work, so you can also work around it by not using the thin launcher.

Can you post a sample in github please?

[gotson]

@dsyer here is the link to my project.

The 2 sub-projects i am building with the aws-adapter are nestor-booker and nestor-core.

The whole project is using gradle multi-projects, all the shared configuration from those 2 projects is stored within the main build.gradle file.

nestor-booker builds fine and generates and -aws.jar that has the correct EnableAutoConfiguration within the spring.factories file.

nestor-core builds fine, but without the extra resources/META-INF/spring.factories file, the generated -aws.jar doesn't have the correct spring.factories.

[gotson]

Indeed the thin jar is not required, i managed to have the same behaviour without the experimental thin jar plugin.
I followed what was done in the aws sample build.gradle, that's why i was using it.

I also managed to reproduce by adding the following dependency to the nestor-booking subproject:

compile 'com.github.spring-data-dynamodb:spring-data-dynamodb:4.5.5'

However when i tried to reproduce on a sample project based on the aws sample i could not get the same broken behaviour, for some reason.

[dsyer]

Why did you do this?

    shadowJar {
        classifier = 'aws'
        dependencies {
            exclude(dependency("org.springframework.cloud:spring-cloud-function-web:${springCloudFunctionVersion}"))
        }
    }

Would it work if you just remove that exclusion? Gradle dependency management can be a bit confusing sometimes, especially for me (as a non Gradle user).

[gotson]

It's in the sample, apparently because of this recommandation.

I just tried changing the task to:

shadowJar {
    classifier = 'aws'
}

It yields the proper spring.factories file inside the -aws.jar.

[gotson]

I'm still confused however as to why the sample has the thin jar plugin, along with this dependency exclusion for the shadowJar task, if none of it is actually required 😕

[dsyer]

The thin launcher is a choice. You can make it or not, it's up to you, but the samples choose it, just to demonstrate that it is possible really. AWS only works with shaded jars, so you need the shadowJar plugin (or the equivalent). I don't know if the shadowJar plugin really works with Spring Boot in general. It seems like it does for the sample in Spring Cloud Function, but that might very well be an accident.

[dsyer]

Some useful comments here: spring-projects/spring-boot#1828.

[dsyer]

Yeah this worked for me (I think). You should check it.

import com.github.jengelman.gradle.plugins.shadow.transformers.*

...

    shadowJar {
        classifier = 'aws'
        dependencies {
            exclude(dependency("org.springframework.cloud:spring-cloud-function-web:${springCloudFunctionVersion}"))
        }
        mergeServiceFiles()
        append 'META-INF/spring.handlers'
        append 'META-INF/spring.schemas'
        append 'META-INF/spring.tooling'
        transform(PropertiesFileTransformer) {
          paths = ['META-INF/spring.factories' ]
          mergeStrategy = "append"
        }
    }

[gotson]

Wow, thanks @dsyer for the clarifications !

Your proposed solution is working great indeed, and the actual content of the spring.factories has much more lines than the single one from my working project (and actually matches what Maven with the shade plugin yields).

I think without your suggested enhancement, it may all come down to the order of dependencies during as for what you get in the shaded jar, explaining the somehow random behaviour i was getting.

[rburgst]

it would be nice to upgrade the sample to use spring boot 2, the newer gradle plugin together with shadowJar prove to create a few problems which are not trivial to solve.

[dsyer]

I’m not a Gradle person. If someone wants to send a PR we can have a look. Please don’t comment on closed issues though.

[eskuai]

any sample maven pom?

[dsyer]

There's a sample here: https://github.com/spring-cloud/spring-cloud-function/tree/master/spring-cloud-function-samples/function-sample-aws. Please don't comment on closed issues. If you have usage questions try stack overflow or gitter.

[eskuai]

Ok, i made multiples test and same problems ... but now i can remove -snapshot from pom
I check multiples versions, and only working if i set environment var FUNCTION_NAME
<spring-cloud-function.version>2.0.0.BUILD-SNAPSHOT</spring-cloud-function.version>
or
<spring-cloud-function.version>2.0.0.M1</spring-cloud-function.version>
I am going to check about ApplicationContextInitializer into springboot class

how convert
@component("morcilla")
public class GreeterFunction implements Function<String, String> {
public String apply(String s) {
return "Hello " + s + ", and welcome to Spring Cloud Function!!!";
}}
to
context.registerBean("function", FunctionRegistration.class,
() -> new FunctionRegistration<Function<String,String>>( xxxx avoid @bean to @component xxxxx )
.type(FunctionType.from(String.class).to(String.class).getType()));

then:
@bean
public Function<String, String> function() {
return new GreeterFunction();
}

@SuppressWarnings("deprecation")
@Override
public void initialize(GenericApplicationContext context) {
	context.registerBean("function", FunctionRegistration.class,
			() -> new FunctionRegistration<Function<String, String>>(function())
					.type(FunctionType.from(String.class).to(String.class).getType()));

}

and ... be sure that FUNCTION_NAME environment var was deleted... run againg ... magic ! works!
"Hello morcilla343, and welcome to Spring Cloud Function!!!"

thank you