Overridden test method: JUnitPlatformFeature unexpectedly failed with a(n) org.junit.platform.commons.JUnitException #541

danthe1st · 2023-11-24T22:43:48Z

Before reporting

This repository should be used to report issues on the Maven or Gradle plugins for GraalVM.
Please report issues which are specific to the Spring Framework or the Micronaut framework to their specific repositories.
Do not report issues with building your specific application, e.g errors which happen at image build time like classes initialized at build time, or missing classes as run time: those are not related to the plugins but problems with configuration. You can refer to the GraalVM native image documentation for available options and the plugins documentation for how to use them with the plugin.

Describe the bug

When overriding an @Test method and without marking the overridden method with @Test, compilation of the test fails with Error: Feature defined by org.graalvm.junit.platform.JUnitPlatformFeature unexpectedly failed with a(n) org.junit.platform.commons.JUnitException. Please report this problem to the authors of org.graalvm.junit.platform.JUnitPlatformFeature..

To Reproduce

Create a Maven Project with JUnit and the Native Maven Plugin
Create a test class with an @Test method
Create another class extending that class and overriding the method
- Do not mark the overridden method in the subclass with @Test

<dependencies>
	<dependency>
		<groupId>org.junit.jupiter</groupId>
		<artifactId>junit-jupiter-api</artifactId>
		<version>5.10.1</version>
	</dependency>
</dependencies>
<build>
	<plugins>
		<plugin>
			<groupId>org.apache.maven.plugins</groupId>
			<artifactId>maven-compiler-plugin</artifactId>
			<version>3.11.0</version>
			<configuration>
				<release>17</release>
			</configuration>
		</plugin>
		<plugin>
			<groupId>org.graalvm.buildtools</groupId>
			<artifactId>native-maven-plugin</artifactId>
			<version>0.9.28</version>
			<extensions>true</extensions>
			<executions>
				<execution>
					<id>build-native</id>
					<goals>
						<goal>compile-no-fork</goal>
					</goals>
					<phase>package</phase>
				</execution>
				<execution>
					<id>test-native</id>
					<goals>
						<goal>test</goal>
					</goals>
					<phase>test</phase>
				</execution>
			</executions>
		</plugin>
		<plugin>
			<groupId>org.apache.maven.plugins</groupId>
			<artifactId>maven-surefire-plugin</artifactId>
			<version>3.2.2</version>
		</plugin>
	</plugins>
</build>

mvn test

import org.junit.jupiter.api.Test;

class SuperTest {
	@Test
	public void test() throws Exception {

	}
}

class SubTest extends SuperTest {
	@Override
	public void test() throws Exception {
	}
}

See this minimal reproducer (CI log here)

Expected behavior

Native compilation of tests succeed and the tests run successfully.

Logs

[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running io.github.danthe1st.junitfeaturefailure.SubTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.048 s -- in io.github.danthe1st.junitfeaturefailure.SubTest
[INFO] Running io.github.danthe1st.junitfeaturefailure.SuperTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 s -- in io.github.danthe1st.junitfeaturefailure.SuperTest
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] 
[INFO] --- native-maven-plugin:0.9.28:test (test-native) @ junit-feature-failure ---
[INFO] ====================
[INFO] Initializing project: junit-feature-failure
[INFO] ====================
[INFO] Found GraalVM installation from GRAALVM_HOME variable.
[INFO] Executing: /opt/hostedtoolcache/graalvm-community-jdk-21.0.1_linux-x64_bin/21.0.1/x64/graalvm-community-openjdk-21.0.1+12.1/bin/native-image -cp /home/runner/work/junit-feature-failure/junit-feature-failure/target/test-classes:/home/runner/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.jar:/home/runner/.m2/repository/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar:/home/runner/.m2/repository/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.jar:/home/runner/.m2/repository/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar:/home/runner/.m2/repository/org/graalvm/buildtools/native-maven-plugin/0.9.28/native-maven-plugin-0.9.28.jar:/home/runner/.m2/repository/org/graalvm/buildtools/junit-platform-native/0.9.28/junit-platform-native-0.9.28.jar:/home/runner/.m2/repository/org/junit/platform/junit-platform-console/1.10.0/junit-platform-console-1.10.0.jar:/home/runner/.m2/repository/org/junit/platform/junit-platform-reporting/1.10.0/junit-platform-reporting-1.10.0.jar:/home/runner/.m2/repository/org/junit/platform/junit-platform-launcher/1.10.0/junit-platform-launcher-1.10.0.jar:/home/runner/.m2/repository/org/junit/platform/junit-platform-engine/1.10.0/junit-platform-engine-1.10.0.jar:/home/runner/.m2/repository/org/junit/platform/junit-platform-commons/1.10.0/junit-platform-commons-1.10.0.jar:/home/runner/.m2/repository/org/junit/jupiter/junit-jupiter/5.10.0/junit-jupiter-5.10.0.jar:/home/runner/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.10.0/junit-jupiter-api-5.10.0.jar:/home/runner/.m2/repository/org/junit/jupiter/junit-jupiter-params/5.10.0/junit-jupiter-params-5.10.0.jar:/home/runner/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.10.0/junit-jupiter-engine-5.10.0.jar:/home/runner/.m2/repository/org/graalvm/buildtools/utils/0.9.28/utils-0.9.28.jar:/home/runner/.m2/repository/org/graalvm/buildtools/graalvm-reachability-metadata/0.9.28/graalvm-reachability-metadata-0.9.28.jar:/home/runner/.m2/repository/org/graalvm/buildtools/junit-platform-native/0.9.28/junit-platform-native-0.9.28.jar --no-fallback -o /home/runner/work/junit-feature-failure/junit-feature-failure/target/native-tests -Djunit.platform.listeners.uid.tracking.output.dir=/home/runner/work/junit-feature-failure/junit-feature-failure/target/test-ids --features=org.graalvm.junit.platform.JUnitPlatformFeature org.graalvm.junit.platform.NativeImageJUnitLauncher
========================================================================================================================
GraalVM Native Image: Generating 'native-tests' (executable)...
========================================================================================================================
For detailed information and explanations on the build output, visit:
https://github.com/oracle/graal/blob/master/docs/reference-manual/native-image/BuildOutput.md
------------------------------------------------------------------------------------------------------------------------
[1/8] Initializing...                                                                                    (3.9s @ 0.07GB)
 Java version: 21.0.1+12, vendor version: GraalVM CE 21.0.1+12.1
 Graal compiler: optimization level: 2, target machine: x86-64-v3
 C compiler: gcc (linux, x86_64, 11.4.0)
 Garbage collector: Serial GC (max heap size: 80% of RAM)
 2 user-specific feature(s):
 - com.oracle.svm.thirdparty.gson.GsonFeature
 - org.graalvm.junit.platform.JUnitPlatformFeature
------------------------------------------------------------------------------------------------------------------------
Build resources:
 - 10.80GB of memory (69.2% of 15.61GB system memory, determined at start)
 - 4 thread(s) (100.0% of 4 available processor(s), determined at start)
[junit-platform-native] Running in 'test listener' mode using files matching pattern [junit-platform-unique-ids*] found in folder [/home/runner/work/junit-feature-failure/junit-feature-failure/target/test-ids] and its subfolders.

Error: Feature defined by org.graalvm.junit.platform.JUnitPlatformFeature unexpectedly failed with a(n) org.junit.platform.commons.JUnitException. Please report this problem to the authors of org.graalvm.junit.platform.JUnitPlatformFeature.
------------------------------------------------------------------------------------------------------------------------
                        0.3s (5.6% of total time) in 12 GCs | Peak RSS: 0.44GB | CPU load: 2.65
========================================================================================================================
Finished generating 'native-tests' in 4.1s.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  13.508 s
[INFO] Finished at: 2023-11-25T22:19:26Z
[INFO] ------------------------------------------------------------------------
Error:  Failed to execute goal org.graalvm.buildtools:native-maven-plugin:0.9.28:test (test-native) on project junit-feature-failure: Execution of /opt/hostedtoolcache/graalvm-community-jdk-21.0.1_linux-x64_bin/21.0.1/x64/graalvm-community-openjdk-21.0.1+12.1/bin/native-image -cp /home/runner/work/junit-feature-failure/junit-feature-failure/target/test-classes:/home/runner/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.jar:/home/runner/.m2/repository/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar:/home/runner/.m2/repository/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.jar:/home/runner/.m2/repository/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar:/home/runner/.m2/repository/org/graalvm/buildtools/native-maven-plugin/0.9.28/native-maven-plugin-0.9.28.jar:/home/runner/.m2/repository/org/graalvm/buildtools/junit-platform-native/0.9.28/junit-platform-native-0.9.28.jar:/home/runner/.m2/repository/org/junit/platform/junit-platform-console/1.10.0/junit-platform-console-1.10.0.jar:/home/runner/.m2/repository/org/junit/platform/junit-platform-reporting/1.10.0/junit-platform-reporting-1.10.0.jar:/home/runner/.m2/repository/org/junit/platform/junit-platform-launcher/1.10.0/junit-platform-launcher-1.10.0.jar:/home/runner/.m2/repository/org/junit/platform/junit-platform-engine/1.10.0/junit-platform-engine-1.10.0.jar:/home/runner/.m2/repository/org/junit/platform/junit-platform-commons/1.10.0/junit-platform-commons-1.10.0.jar:/home/runner/.m2/repository/org/junit/jupiter/junit-jupiter/5.10.0/junit-jupiter-5.10.0.jar:/home/runner/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.10.0/junit-jupiter-api-5.10.0.jar:/home/runner/.m2/repository/org/junit/jupiter/junit-jupiter-params/5.10.0/junit-jupiter-params-5.10.0.jar:/home/runner/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.10.0/junit-jupiter-engine-5.10.0.jar:/home/runner/.m2/repository/org/graalvm/buildtools/utils/0.9.28/utils-0.9.28.jar:/home/runner/.m2/repository/org/graalvm/buildtools/graalvm-reachability-metadata/0.9.28/graalvm-reachability-metadata-0.9.28.jar:/home/runner/.m2/repository/org/graalvm/buildtools/junit-platform-native/0.9.28/junit-platform-native-0.9.28.jar --no-fallback -o /home/runner/work/junit-feature-failure/junit-feature-failure/target/native-tests -Djunit.platform.listeners.uid.tracking.output.dir=/home/runner/work/junit-feature-failure/junit-feature-failure/target/test-ids --features=org.graalvm.junit.platform.JUnitPlatformFeature org.graalvm.junit.platform.NativeImageJUnitLauncher returned non-zero result -> [Help 1]
Error:  
Error:  To see the full stack trace of the errors, re-run Maven with the -e switch.
Error:  Re-run Maven using the -X switch to enable full debug logging.
Error:  
Error:  For more information about the errors and possible solutions, please read the following articles:
Error:  [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Error: Process completed with exit code 1.

Full logs here

System Info (please complete the following information):

OS: Linux
GraalVM Version: graalvm-community-jdk-21.0.1_linux-x64_bin
Java Version 21
Plugin version native-maven-plugin:0.9.28

Additional context

Full reproducer here

The text was updated successfully, but these errors were encountered:

danthe1st · 2023-11-25T00:48:39Z

I am reporting this here since JUnitPlatformFeature seems to be part of this repository.

danthe1st · 2024-01-30T19:15:06Z

@dnestoro Is there a chance at getting this triaged?

melix · 2024-02-01T12:21:17Z

Possibly fixed by #565 although I have not checked.

danthe1st · 2024-02-02T16:36:36Z

Possibly fixed by #565 although I have not checked.

~~It seems to still be the case in 0.10.0. I have updated the reproducer.~~ nvm that PR isn't merged yet

danthe1st · 2024-02-20T11:37:41Z

Now that 0.10.1 has been released with #545, I can still reproduce this (GH Actions)
@melix

danthe1st · 2024-05-25T07:59:27Z

Still happens with 0.10.2.

stevenschlansker · 2024-10-15T18:07:14Z

We're seeing this exception on 0.10.3 as well.

danthe1st · 2024-10-15T20:11:01Z

@stevenschlansker As a workaround, you can mark the overridden method with @Test.

stevenschlansker · 2024-10-16T16:51:34Z

@danthe1st , thanks for the suggestion but ... which test case? The error message does not print out which test case causes the failure, and we have thousands of tests :( I suppose I can narrow the search to a test class that extends another, but still. If this issue really won't be fixed I will eventually invest the time to look at doing so... but given the number of users reporting this, I will hope for a fix in the next release for now :)

danthe1st · 2024-10-16T18:13:45Z

@stevenschlansker
Yeah I had the same issue. Essentially look at overridden test methods that aren't annotated with @Test. You can also write some scripts for automating that.

danthe1st added the bug Something isn't working label Nov 24, 2023

danthe1st changed the title ~~JUnitPlatformFeature unexpectedly failed with a(n) org.junit.platform.commons.JUnitException~~ @Disabled with native tests: JUnitPlatformFeature unexpectedly failed with a(n) org.junit.platform.commons.JUnitException Nov 24, 2023

danthe1st changed the title ~~@Disabled with native tests: JUnitPlatformFeature unexpectedly failed with a(n) org.junit.platform.commons.JUnitException~~ Overridden test: JUnitPlatformFeature unexpectedly failed with a(n) org.junit.platform.commons.JUnitException Nov 25, 2023

danthe1st changed the title ~~Overridden test: JUnitPlatformFeature unexpectedly failed with a(n) org.junit.platform.commons.JUnitException~~ Overridden test method: JUnitPlatformFeature unexpectedly failed with a(n) org.junit.platform.commons.JUnitException Nov 25, 2023

stevenschlansker mentioned this issue Oct 15, 2024

Run CI with GraalVM jdbi/jdbi#2476

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Overridden test method: JUnitPlatformFeature unexpectedly failed with a(n) org.junit.platform.commons.JUnitException #541

Overridden test method: JUnitPlatformFeature unexpectedly failed with a(n) org.junit.platform.commons.JUnitException #541

danthe1st commented Nov 24, 2023 •

edited

Loading

danthe1st commented Nov 25, 2023

danthe1st commented Jan 30, 2024

melix commented Feb 1, 2024

danthe1st commented Feb 2, 2024 •

edited

Loading

danthe1st commented Feb 20, 2024

danthe1st commented May 25, 2024

stevenschlansker commented Oct 15, 2024

danthe1st commented Oct 15, 2024

stevenschlansker commented Oct 16, 2024

danthe1st commented Oct 16, 2024

Overridden test method: JUnitPlatformFeature unexpectedly failed with a(n) org.junit.platform.commons.JUnitException #541

Overridden test method: JUnitPlatformFeature unexpectedly failed with a(n) org.junit.platform.commons.JUnitException #541

Comments

danthe1st commented Nov 24, 2023 • edited Loading

danthe1st commented Nov 25, 2023

danthe1st commented Jan 30, 2024

melix commented Feb 1, 2024

danthe1st commented Feb 2, 2024 • edited Loading

danthe1st commented Feb 20, 2024

danthe1st commented May 25, 2024

stevenschlansker commented Oct 15, 2024

danthe1st commented Oct 15, 2024

stevenschlansker commented Oct 16, 2024

danthe1st commented Oct 16, 2024

danthe1st commented Nov 24, 2023 •

edited

Loading

danthe1st commented Feb 2, 2024 •

edited

Loading