[native-image] NPE when building image serialization-config involving invalid customTargetConstructorClass #3521

aldettinger · 2021-06-28T08:22:42Z

Describe the issue
This issue occurs in the context of building a native executable involving JBoss River serialization. From #3156, it looks that one needs to add an entry in serialization-config.json for each direct call to jdk.internal.reflect.ReflectionFactory#newConstructorForSerialization(Class<?>, Constructor<?>). In the case at hand, we end up with a serialization-configs with 400 entries but the build fails with a java.lang.NullPointerException.

Steps to reproduce the issue
I was not able to setup a minimal reproducer but steps below should do the job

git clone -b CAMEL-QUARKUS-2107 https://github.com/aldettinger/camel-quarkus.git
mvn clean install -Dquickly
cd extensions-core
mvn clean install
cd ../integration-tests/redis/
mvn clean integration-test -P native

Describe GraalVM and your environment:
GraalVM 21.1.0 Java 11 CE (Java Version 11.0.11+8-jvmci-21.1-b05)
openjdk version "11.0.9.1" 2020-11-04
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.9.1+1)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.9.1+1, mixed mode)
Linux aldettinger.remote.csb 3.10.0-1160.25.1.el7.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux

More details

[INFO] [io.quarkus.deployment.pkg.steps.JarResultBuildStep] Building native image source jar: /home/agallice/dev/projects/camel-quarkus-upstream/integration-tests/redis/target/camel-quarkus-integration-test-redis-2.0.0-SNAPSHOT-native-image-source-jar/camel-quarkus-integration-test-redis-2.0.0-SNAPSHOT-runner.jar
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Building native image from /home/agallice/dev/projects/camel-quarkus-upstream/integration-tests/redis/target/camel-quarkus-integration-test-redis-2.0.0-SNAPSHOT-native-image-source-jar/camel-quarkus-integration-test-redis-2.0.0-SNAPSHOT-runner.jar
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on GraalVM 21.1.0 Java 11 CE (Java Version 11.0.11+8-jvmci-21.1-b05)
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildRunner] /home/agallice/dev/graalvm/graalvm-ce-java11-21.1.0/bin/native-image -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=3 -J-Duser.language=en -J-Duser.country=US -J-Dfile.encoding=UTF-8 -H:SerializationConfigurationResources=serialization-config.json --initialize-at-build-time= -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy\$BySpaceAndTime -H:+JNI -H:+AllowFoldMethods -jar camel-quarkus-integration-test-redis-2.0.0-SNAPSHOT-runner.jar -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:-AddAllCharsets -H:EnableURLProtocols=http -H:-UseServiceLoaderFeature -H:+StackTrace camel-quarkus-integration-test-redis-2.0.0-SNAPSHOT-runner
[camel-quarkus-integration-test-redis-2.0.0-SNAPSHOT-runner:21202]    classlist:   5,268.89 ms,  1.19 GB
[camel-quarkus-integration-test-redis-2.0.0-SNAPSHOT-runner:21202]        (cap):     503.37 ms,  1.19 GB
[camel-quarkus-integration-test-redis-2.0.0-SNAPSHOT-runner:21202]        setup:   1,263.63 ms,  1.19 GB
Fatal error:java.lang.NullPointerException
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:603)
	at java.base/java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1006)
	at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:499)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:370)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:531)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:119)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus.main(NativeImageGeneratorRunner.java:568)
Caused by: java.lang.NullPointerException
	at java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
	at java.base/java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1541)
	at com.oracle.svm.reflect.serialize.SerializationSupport.addConstructorAccessor(SerializationSupport.java:119)
	at com.oracle.svm.reflect.serialize.hosted.SerializationBuilder.addConstructorAccessor(SerializationFeature.java:313)
	at com.oracle.svm.reflect.serialize.hosted.SerializationFeature.lambda$duringSetup$1(SerializationFeature.java:115)
	at com.oracle.svm.core.configure.SerializationConfigurationParser.parseAndRegister(SerializationConfigurationParser.java:54)
	at com.oracle.svm.hosted.config.ConfigurationParserUtils.doParseAndRegister(ConfigurationParserUtils.java:133)
	at com.oracle.svm.hosted.config.ConfigurationParserUtils.lambda$parseAndRegisterConfigurations$3(ConfigurationParserUtils.java:120)
	at java.base/java.util.stream.ReferencePipeline$4$1.accept(ReferencePipeline.java:212)
	at com.oracle.svm.hosted.config.ConfigurationParserUtils$1.tryAdvance(ConfigurationParserUtils.java:109)
	at java.base/java.util.Spliterator.forEachRemaining(Spliterator.java:326)
	at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658)
	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:274)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
	at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:312)
	at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.IntPipeline.reduce(IntPipeline.java:491)
	at java.base/java.util.stream.IntPipeline.sum(IntPipeline.java:449)
	at com.oracle.svm.hosted.config.ConfigurationParserUtils.parseAndRegisterConfigurations(ConfigurationParserUtils.java:126)
	at com.oracle.svm.reflect.serialize.hosted.SerializationFeature.duringSetup(SerializationFeature.java:122)
	at com.oracle.svm.hosted.NativeImageGenerator.lambda$setupNativeImage$18(NativeImageGenerator.java:915)
	at com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:71)
	at com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:915)
	at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:580)
	at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$2(NativeImageGenerator.java:495)
	at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Error: Image build request failed with exit status 1
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------

The text was updated successfully, but these errors were encountered:

munishchouhan · 2021-06-30T05:34:16Z

@aldettinger thanks for reporting the issue, we will check it out and get back to you

munishchouhan · 2021-07-01T06:32:50Z

@aldettinger is it possible to provide a simple reprodcuer or a sample code?

aldettinger · 2021-07-01T07:46:11Z

I'm trying to setup a simple reproducer but not succeeding yet. As of today, I could only reproduce on the CAMEL-QUARKUS-2107 branch as described in steps to reproduce. Are you able to reproduce on that branch ? I should be able to assist if you have issues building it.

aldettinger · 2021-07-01T08:03:38Z

@mcraj017 The steps to reproduce section was dubious and I have corrected it. Locally, I could rebuild and reproduce from there with java 11, maven 3.6.3 and native-image 21.1.0. It arguably takes a while to build and would be good to shrink the reproducer but at this stage I've not succeeded.

munishchouhan · 2021-07-05T05:34:59Z

@aldettinger I tired but got error at the end, I will give a try again, thanks

aldettinger · 2021-07-05T07:24:05Z

@mcraj017 Sure, in case of persisting issue, I'm happy to help.

aldettinger · 2021-07-07T13:00:05Z

@mcraj017 I was able to shrink to a more minimal reproducer. Does it reproduce on your side too ?

olpaw · 2021-07-13T13:44:12Z

@mcraj017 I was able to shrink to a more minimal reproducer. Does it reproduce on your side too ?

@aldettinger many thanks for the reduced reproducer. I can confirm that it allows us to reproduce the issue.

olpaw · 2021-07-15T10:00:50Z

@aldettinger what you are doing in serialization-config.json makes no sense:

{
	"name": "java.util.LinkedHashMap",
	"customTargetConstructorClass": "java.util.LinkedHashMap"
}

customTargetConstructorClass is only to be used for special corner-cases. Unfortunately some frameworks directly call the JDK internal method jdk.internal.reflect.ReflectionFactory#newConstructorForSerialization(java.lang.Class<?>, java.lang.reflect.Constructor<?>) via reflection where they pass in a custom java.lang.reflect.Constructor<?> to be used to initialize the instance for deserialization.

Usually this is not necessary as serialization infrastructure decides itself which constructor should be used to initialize the object that needs to be deserialized. The constructor that will be used is specified in the serialization specification chapter 3.1 (11.a.) https://docs.oracle.com/javase/8/docs/platform/serialization/spec/input.html

For regular serialization use cases the "customTargetConstructorClass" attribute should not be used in serialization-config.

That said, if a serialization-config entry like the one you provided in your reproducer is used, native-image should not run into a NullPointerException. I will make sure that we provide a proper error message in this case instead.

olpaw · 2021-07-15T11:58:14Z

BTW, #3581 will add agent support for generating serialization-configs where customTargetConstructorClass-use will be correctly detected. This should make JBoss River serialization work with agent based serialization-config generation cc @jovanstevanovic

aldettinger · 2021-07-28T13:53:21Z

@olpaw Many thanks for taking a look.

Ok, so having customTargetConstructorClass == serializationTargetClass is not a valid configuration from native-image point of view. In such a case, the standard reflection de-serialization mechanism is expected to automatically resolve the right constructor.

I've debugged the JBoss River case at hand and it happens that ReflectionFactory#newConstructorForSerialization(java.lang.Class<?>, java.lang.reflect.Constructor<?>)
is called with such an argument pair. So, indeed the agent support will help a lot in order to generate the right serialization configuration.

aldettinger added the bug label Jun 28, 2021

munishchouhan self-assigned this Jun 30, 2021

munishchouhan added native-image quarkus quarkus related issue labels Jun 30, 2021

munishchouhan assigned christianwimmer and unassigned munishchouhan Jul 9, 2021

olpaw self-assigned this Jul 13, 2021

olpaw changed the title ~~[native-image] java.lang.NullPointerException when building a native executable with a serialization-config involving customTargetConstructorClass~~ [native-image] NPE when building image serialization-config involving invalid customTargetConstructorClass Jul 15, 2021

olpaw linked a pull request Jul 15, 2021 that will close this issue

[GR-32549] Fix NPE when using serialization-config involving customTargetConstructorClass. #3593

Merged

graalvmbot closed this as completed in #3593 Jul 16, 2021

aldettinger mentioned this issue May 25, 2022

[native-image] java.lang.NoSuchMethodException: java.util.Collections$UnmodifiableMap$UnmodifiableEntrySet.<init>() #4600

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[native-image] NPE when building image serialization-config involving invalid customTargetConstructorClass #3521

[native-image] NPE when building image serialization-config involving invalid customTargetConstructorClass #3521

aldettinger commented Jun 28, 2021 •

edited

Loading

munishchouhan commented Jun 30, 2021

munishchouhan commented Jul 1, 2021

aldettinger commented Jul 1, 2021

aldettinger commented Jul 1, 2021

munishchouhan commented Jul 5, 2021

aldettinger commented Jul 5, 2021

aldettinger commented Jul 7, 2021

olpaw commented Jul 13, 2021

olpaw commented Jul 15, 2021 •

edited

Loading

olpaw commented Jul 15, 2021 •

edited

Loading

aldettinger commented Jul 28, 2021

[native-image] NPE when building image serialization-config involving invalid customTargetConstructorClass #3521

[native-image] NPE when building image serialization-config involving invalid customTargetConstructorClass #3521

Comments

aldettinger commented Jun 28, 2021 • edited Loading

munishchouhan commented Jun 30, 2021

munishchouhan commented Jul 1, 2021

aldettinger commented Jul 1, 2021

aldettinger commented Jul 1, 2021

munishchouhan commented Jul 5, 2021

aldettinger commented Jul 5, 2021

aldettinger commented Jul 7, 2021

olpaw commented Jul 13, 2021

olpaw commented Jul 15, 2021 • edited Loading

olpaw commented Jul 15, 2021 • edited Loading

aldettinger commented Jul 28, 2021

aldettinger commented Jun 28, 2021 •

edited

Loading

olpaw commented Jul 15, 2021 •

edited

Loading

olpaw commented Jul 15, 2021 •

edited

Loading