Additional metadata not merged with boot autoconfigure jar in classpath

[jvalkeal]

With gradle(should also happen with maven) additional-spring-configuration-metadata.json is not merged into spring-configuration-metadata.json which seems to be caused by a multiple files in a tool classpath.

I added some logging:

private ConfigurationMetadata mergeManualMetadata(ConfigurationMetadata metadata) {
  try { 
          FileObject manualMetadata = this.processingEnv.getFiler().getResource(
                          StandardLocation.CLASS_PATH, "",
                          "META-INF/additional-spring-configuration-metadata.json");
          System.out.println("XXX1: " + manualMetadata);
          System.out.println("XXX2: " + manualMetadata.toUri());
          System.out.println("XXX3: " + manualMetadata.toUri().getScheme());
          if (!"file".equals(manualMetadata.toUri().getScheme())) {
                  // We only want local files, not any classpath jars
                  return metadata;
          }

When compiling boot itself, I see expected shown below:

[INFO] Compiling 191 source files to /home/jvalkealahti/repos/jvalkeal/spring-boot/spring-boot/target/classes
XXX1: com.sun.tools.javac.processing.JavacFiler$FilerInputFileObject@cce92a5
XXX2: file:/home/jvalkealahti/repos/jvalkeal/spring-boot/spring-boot/target/classes/META-INF/additional-spring-configuration-metadata.json
XXX3: file

When I added bits to SHDP boot module, I see this:

:spring-data-hadoop-boot:compileJava
warning: [options] bootstrap class path not set in conjunction with -source 1.6
XXX1: com.sun.tools.javac.processing.JavacFiler$FilerInputFileObject@4167292
XXX2: jar:file:/home/jvalkealahti/.m2/repository/org/springframework/boot/spring-boot-autoconfigure/1.2.1.BUILD-SNAPSHOT/spring-boot-autoconfigure-1.2.1.BUILD-SNAPSHOT.jar!/META-INF/additional-spring-configuration-metadata.json
XXX3: jar

Scheme is jar so no metadata gets merged. Tests jar is seeing this json with a file scheme and stuff is actually merged correctly in this case. I guess in case of tests.jar location of additional-spring-configuration-metadata.json just happens to be correct one returned by a JavacFileManager.

:spring-data-hadoop-boot:compileTestJava
warning: [options] bootstrap class path not set in conjunction with -source 1.6
XXX1: com.sun.tools.javac.processing.JavacFiler$FilerInputFileObject@4f7f7eba
XXX2: file:/home/jvalkealahti/repos/jvalkeal/spring-hadoop/spring-hadoop-boot/build/resources/main/META-INF/additional-spring-configuration-metadata.json
XXX3: file

Checking JavacFileManager sources at
http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/com/sun/tools/javac/util/JavacFileManager.java#1097

} else {
    Iterable<? extends File> path = paths.getPathForLocation(location);
    dir = null;
    for (File f: path) {
        dir = f;
        break;
    }
}

First path is returned whatever it is. spring-hadoop-boot depends on spring-boot-autoconfigure so second additional-spring-configuration-metadata.json is always there. I guess this is only reliable if classpath contains only one additional-spring-configuration-metadata.json file.

[jvalkeal]

Something I didn't try but thought about, instead of using javax.tools.StandardLocation.CLASS_PATH, could we use javax.tools.StandardLocation.SOURCE_PATH? If this is only used at a compile time could it be better to find json file by source path.

[jvalkeal]

@philwebb I think this is not yet ok. Just tried from master with additional logging:

:spring-data-hadoop-boot:compileJava
warning: [options] bootstrap class path not set in conjunction with -source 1.6
XXX1: com.sun.tools.javac.processing.JavacFiler$FilerOutputJavaFileObject@2ace94ff
XXX2: file:/home/jvalkealahti/repos/jvalkeal/spring-hadoop/spring-hadoop-boot/build/classes/main/META-INF/additional-spring-configuration-metadata.json
XXX3: file
XXX4: java.io.FileNotFoundException: /home/jvalkealahti/repos/jvalkeal/spring-hadoop/spring-hadoop-boot/build/classes/main/META-INF/additional-spring-configuration-metadata.json (No such file or directory)
warning: Unable to merge additional-spring-configuration-metadata.json
1 warning
:spring-data-hadoop-boot:processResources
:spring-data-hadoop-boot:classes
:spring-data-hadoop-boot:jar
:spring-data-hadoop-boot:sourcesJar

File is in src/main/resources/META-INF/additional-spring-configuration-metadata.json and would be copied under build/resources/main/META-INF/additional-spring-configuration-metadata.json. Thought this copy doesn't happen until processResources is called which in my build happens after compileJava.

[philwebb]

@jvalkeal Do you you have a branch with this code committed somewhere that I can use to test?

[jvalkeal]

@philwebb yeah, https://github.com/jvalkeal/spring-hadoop/tree/SHDP-452-work1 which uses 1.2.1.BUILD-SNAPSHOT with mavenLocal(). Do something like:

./gradlew :spring-data-hadoop-boot:clean :spring-data-hadoop-boot:compileJava -x test

[philwebb]

Cheers. I think I have a fix now.

[jvalkeal]

@philwebb just a note here. This now works but I need to tell gradle to run processResources before compileJava. Otherwise json is not in place when compile / annotation processor kicks in.

compileJava.dependsOn(processResources)

Maybe add a little note section to docs?

[philwebb]

Arrrghhh!

[philwebb]

I've raised #2316 :(

[btiernay]

I'm having an issue with my properties not resolving in a shadowJar using Gradle. Could this be the root cause? I have nothing merging spring-configuration-metadata.json, etc.

[wilkinsona]

@btiernay I doubt it. The metadata described in this issue isn't used at runtime. It's used by IDEs to provide auto-completion

[btiernay]

@wilkinsona turns out this was related to #1828. Cheers!