[DSIP-27][Task Plugin] Some improvements of JAVA task plugin

docs/docs/en/guide/task/java.md

@@ -1,6 +1,6 @@
 # Overview
 
-This node is for executing java-type tasks and supports using files and jar packages as program entries.
+This node is used to execute Java tasks and supports using both fat and normal JAR packages as program entry points.
 
 # Create Tasks
 
@@ -20,24 +20,37 @@ This node is for executing java-type tasks and supports using files and jar pack
 | Module Path        | pick Java 9 + 's modularity feature, put all resources into-module-path, and require that the JDK version in your worker supports modularity. |
 | Main Parameter     | Java program main method entry parameter.                                                                                                     |
 | Java VM Parameters | JVM startup parameters.                                                                                                                       |
-| Script             | You need to write Java code if you use the Java run type. The public class must exist in the code without writing a package statement.        |
+| Main Package       | Select the main program package to run the application.                                                                                       |
 | Resources          | External JAR packages or other resource files that are added to the classpath or module path and can be easily retrieved in your JAVA script. |
 
 ## Example
 
-Java type tasks have two modes of execution, here is a demonstration of executing tasks in Java mode.
+There are two execution modes for Java task types, which will be demonstrated separately here.
 
 The main configuration parameters are as follows:
 - Run Type
 - Module Path
 - Main Parameters
 - Java VM Parameters
-- Script
+- Main Package
+- Resources
 
-![java_task](../../../../img/tasks/demo/java_task02.png)
+As shown in the figure.
+
+- FAT_JAR
+
+![java_task](../../../../img/tasks/demo/java_fat.png)
+
+Since a fat-type JAR includes both dependencies and code within the same JAR, you only need to select this one JAR.
+
+- NORMAL_JAR
+
+![java_task](../../../../img/tasks/demo/java_normal.png)
+
+normal1.jar serves as the entry point for the program, while normal2.jar is a required dependency. You need to specify the program's entry point using the main program package and select all dependency and entry JAR files in the resource files to ensure correct execution.normal1.jar serves as the entry point for the program, while normal2.jar is a required dependency. You need to specify the program's entry point using the main program package and select all dependency and entry JAR files in the resource files to ensure correct execution.
 
 ## Note
 
-When you run the task in JAVA execution mode, the public class must exist in the code, and you could omit writing a package statement.
+When using these two execution modes, you must select both the main program package and the corresponding resource files; otherwise, the task will fail immediately. This is because the main program package determines where the program starts running, and after selecting the resource files, they will be included when the program is executed.
 
 For security reasons, when executing JAVA tasks, please use the environment management module to configure the JDK environment, such as `JAVA_HOME` and other environment variables.

docs/docs/zh/guide/task/java.md

@@ -2,7 +2,7 @@
 
 ## 综述
 
-该节点用于执行 java 类型的任务，支持使用单文件和jar包作为程序入口。
+该节点用于执行 java 类型的任务，支持使用fat类型和normal类型的jar包作为程序入口。
 
 ## 创建任务
 
@@ -22,25 +22,38 @@
 | 模块路径     | 开启使用JAVA9+的模块化特性，把所有资源放入--module-path中，要求您的worker中的JDK版本支持模块化 |
 | 主程序参数    | 作为普通Java程序main方法入口参数                                          |
 | 虚拟机参数    | 配置启动虚拟机参数                                                     |
-| 脚本       | 若使用JAVA运行类型则需要编写JAVA代码。代码中必须存在public类，不用写package语句            |
+| 主程序包     | 选择要运行程序的主程序包                                                  |
 | 资源       | 可以是外部JAR包也可以是其他资源文件，它们都会被加入到类路径或模块路径中，您可以在自己的JAVA脚本中轻松获取      |
 
 ## 任务样例
 
-java任务类型有两种运行模式，这里以JAVA模式为例进行演示。
+java任务类型有两种运行模式，这里进行分别进行演示。
 
 主要配置参数如下：
 
 - 运行类型
 - 模块路径
 - 主程序参数
 - 虚拟机参数
-- 脚本文件
+- 主程序包
+- 资源文件
 
-![java_task](../../../../img/tasks/demo/java_task02.png)
+如图所示
+
+- FAT_JAR类型
+
+![java_task](../../../../img/tasks/demo/java_fat.png)
+
+因为fat类型的jar，依赖和代码都在同一个jar中，则只需选择这一个jar即可
+
+- NORMAL_JAR类型
+
+![java_task](../../../../img/tasks/demo/java_normal.png)
+
+normal1.jar是程序运行的入口，normal2.jar是所需的依赖，需要用主程序包指定程序的入口，并且在资源文件中选择所有依赖和程序入口jar文件，才能正确运行
 
 ## 注意事项
 
-使用JAVA运行类型时代码中必须存在public类，可以不写package语句
+使用这两种运行类型，都必须选择主程序包和对应的资源文件，否则任务会直接失败。因为主程序包是选择程序从哪里开始运行，资源文件选择文件之后，运行程序时，资源文件才能被加入其中
 
 基于安全原因，执行JAVA任务时，请使用环境管理功能配置JDK环境，例如`JAVA_HOME`等环境变量

dolphinscheduler-e2e/dolphinscheduler-e2e-case/src/test/java/org/apache/dolphinscheduler/e2e/cases/WorkflowJavaTaskE2ETest.java

@@ -29,11 +29,34 @@
 import org.apache.dolphinscheduler.e2e.pages.project.workflow.WorkflowForm;
 import org.apache.dolphinscheduler.e2e.pages.project.workflow.WorkflowInstanceTab;
 import org.apache.dolphinscheduler.e2e.pages.project.workflow.task.JavaTaskForm;
+import org.apache.dolphinscheduler.e2e.pages.resource.FileManagePage;
+import org.apache.dolphinscheduler.e2e.pages.resource.ResourcePage;
 import org.apache.dolphinscheduler.e2e.pages.security.EnvironmentPage;
 import org.apache.dolphinscheduler.e2e.pages.security.SecurityPage;
 import org.apache.dolphinscheduler.e2e.pages.security.TenantPage;
 import org.apache.dolphinscheduler.e2e.pages.security.UserPage;
 
+import java.io.File;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.net.URL;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.time.Duration;
+import java.util.Arrays;
+import java.util.List;
+import java.util.jar.Attributes;
+import java.util.jar.JarEntry;
+import java.util.jar.JarOutputStream;
+import java.util.jar.Manifest;
+
+import javax.tools.JavaCompiler;
+import javax.tools.JavaFileObject;
+import javax.tools.StandardJavaFileManager;
+import javax.tools.ToolProvider;
+
+import lombok.extern.slf4j.Slf4j;
+
 import org.junit.jupiter.api.AfterAll;
 import org.junit.jupiter.api.BeforeAll;
 import org.junit.jupiter.api.Order;
@@ -42,16 +65,20 @@
 import org.openqa.selenium.By;
 import org.openqa.selenium.remote.RemoteWebDriver;
 import org.openqa.selenium.support.ui.ExpectedConditions;
+import org.openqa.selenium.support.ui.WebDriverWait;
 import org.testcontainers.shaded.org.awaitility.Awaitility;
 
 @DolphinScheduler(composeFiles = "docker/basic/docker-compose.yaml")
 @DisableIfTestFails
+@Slf4j
 public class WorkflowJavaTaskE2ETest {
 
-    private static final String project = "test-workflow-1";
+    private static final String project = "test-workflow";
 
     private static final String workflow = "test-workflow-1";
 
+    private static final String workflow2 = "test-workflow-2";
+
     private static final String user = "admin";
 
     private static final String password = "dolphinscheduler123";
@@ -70,16 +97,110 @@ public class WorkflowJavaTaskE2ETest {
 
     private static final String environmentWorkerGroup = "default";
 
-    private static final String javaContent = "public class Test {" +
-            "    public static void main(String[] args) {" +
-            "        System.out.println(\"hello world\");" +
-            "    }" +
-            "}";
+    private static final String filePath = "/tmp";
 
     private static RemoteWebDriver browser;
 
+    private static void createJar(String className, String classFilePath, String entryName, String mainPackage,
+                                  String jarName) {
+
+        String jarFilePath = "/tmp/" + jarName;
+
+        Manifest manifest = new Manifest();
+        manifest.getMainAttributes().put(Attributes.Name.MANIFEST_VERSION, "1.0");
+        manifest.getMainAttributes().put(Attributes.Name.MAIN_CLASS, mainPackage);
+
+        try (
+                FileOutputStream fos = new FileOutputStream(jarFilePath);
+                JarOutputStream jos = new JarOutputStream(fos, manifest)) {
+            Path path = new File(classFilePath + className).toPath();
+            JarEntry entry = new JarEntry(entryName);
+            jos.putNextEntry(entry);
+            byte[] bytes = Files.readAllBytes(path);
+            jos.write(bytes, 0, bytes.length);
+            jos.closeEntry();
+        } catch (IOException e) {
+            log.error("create jar failed:", e);
+        }
+
+    }
+    private static void getJar() {
+        String classPath = "/tmp/common/";
+        compileJavaFile("common/Fat.java");
+        compileJavaFile("common/Normal1.java");
+        compileJavaFile("common/Normal2.java");
+        createJar("Fat.class", classPath,
+                "common/Fat.class",
+                "common.Fat",
+                "fat.jar");
+        createJar("Normal1.class",
+                classPath,
+                "common/Normal1.class",
+                "common.Normal1",
+                "normal1.jar");
+        createJar("Normal2.class",
+                classPath,
+                "common/Normal2.class",
+                "common.Normal2",
+                "normal2.jar");
+
+    }
+
+    public static void compileJavaFile(String sourceFilePath) {
+
+        JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
+        if (compiler == null) {
+            log.error("Cannot find the system Java compiler.", new IllegalStateException());
+        }
+
+        ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
+        URL resourceUrl = classLoader.getResource(sourceFilePath);
+        String absolutePath = "";
+
+        try {
+            File resourceFile = new File(resourceUrl.toURI());
+            absolutePath = resourceFile.getAbsolutePath();
+        } catch (Exception e) {
+            log.error(" java file cannot find:", e);
+        }
+
+        String outputDirPath = "/tmp";
+
+        try (StandardJavaFileManager fileManager = compiler.getStandardFileManager(null, null, null)) {
+
+            File sourceFile = new File(absolutePath);
+            if (!sourceFile.exists()) {
+                log.error("java file not exist", new IllegalArgumentException());
+            }
+
+            Iterable<? extends JavaFileObject> compilationUnits =
+                    fileManager.getJavaFileObjectsFromFiles(Arrays.asList(sourceFile));
+
+            List<String> options = Arrays.asList(
+                    "--release", "8",
+                    "-d", outputDirPath);
+
+            JavaCompiler.CompilationTask task = compiler.getTask(
+                    null,
+                    fileManager,
+                    null,
+                    options,
+                    null,
+                    compilationUnits);
+
+            boolean success = task.call();
+
+            if (!success) {
+                throw new RuntimeException("Compilation failed.");
+            }
+        } catch (IOException e) {
+            log.error("compile java file failed:", e);
+        }
+    }
+
     @BeforeAll
     public static void setup() {
+        getJar();
         UserPage userPage = new LoginPage(browser)
                 .login(user, password)
                 .goToNav(SecurityPage.class)
@@ -97,6 +218,11 @@ public static void setup() {
         userPage.update(user, user, email, phone, tenant)
                 .goToNav(ProjectPage.class)
                 .create(project);
+
+        ProjectPage projectPage = new ProjectPage(browser);
+        Awaitility.await().untilAsserted(() -> assertThat(projectPage.projectList())
+                .as("The project list should include newly created projects.")
+                .anyMatch(it -> it.getText().contains(project)));
     }
 
     @AfterAll
@@ -121,35 +247,47 @@ public static void cleanup() {
 
     @Test
     @Order(1)
-    void testCreateWorkflow() {
-        WorkflowDefinitionTab workflowDefinitionPage =
-                new ProjectPage(browser)
-                        .goTo(project)
-                        .goToTab(WorkflowDefinitionTab.class);
+    void testCreateFatWorkflow() {
+        FileManagePage file = new NavBarPage(browser)
+                .goToNav(ResourcePage.class)
+                .goToTab(FileManagePage.class)
+                .uploadFile(filePath + "/fat.jar");
+
+        WebDriverWait wait = WebDriverWaitFactory.createWebDriverWait(browser);
 
-        workflowDefinitionPage
-                .createWorkflow()
-                .<JavaTaskForm>addTask(WorkflowForm.TaskType.JAVA)
-                .script(javaContent)
+        wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//span[text()='fat.jar']")));
+
+        ProjectPage projectPage = new NavBarPage(browser)
+                .goToNav(ProjectPage.class);
+
+        wait.until(ExpectedConditions.visibilityOfAllElements(projectPage.projectList()));
+
+        WorkflowDefinitionTab workflowDefinitionPage = projectPage
+                .goTo(project)
+                .goToTab(WorkflowDefinitionTab.class);
+
+        WorkflowForm workflow1 = workflowDefinitionPage.createWorkflow();
+        workflow1.<JavaTaskForm>addTask(WorkflowForm.TaskType.JAVA)
+                .selectRunType("FAT_JAR")
+                .selectMainPackage("fat.jar")
+                .selectJavaResource("fat.jar")
                 .name("test-1")
-                .addParam("today", "${system.datetime}")
                 .selectEnv(environmentName)
                 .submit()
                 .submit()
                 .name(workflow)
-                .addGlobalParam("global_param", "hello world")
                 .submit();
 
         Awaitility.await().untilAsserted(() -> assertThat(workflowDefinitionPage.workflowList())
                 .as("Workflow list should contain newly-created workflow")
-                .anyMatch(
-                        it -> it.getText().contains(workflow)));
+                .anyMatch(it -> it.getText().contains(workflow)));
+
         workflowDefinitionPage.publish(workflow);
     }
 
     @Test
     @Order(30)
-    void testRunWorkflow() {
+    void testRunFatWorkflow() {
         final ProjectDetailPage projectPage =
                 new ProjectPage(browser)
                         .goToNav(ProjectPage.class)
@@ -163,17 +301,21 @@ void testRunWorkflow() {
                 .run(workflow)
                 .submit();
 
-        Awaitility.await().untilAsserted(() -> {
-            browser.navigate().refresh();
+        Awaitility.await()
+                .atMost(Duration.ofMinutes(5))
+                .untilAsserted(() -> {
+                    browser.navigate().refresh();
 
-            final WorkflowInstanceTab.Row row = projectPage
-                    .goToTab(WorkflowInstanceTab.class)
-                    .instances()
-                    .iterator()
-                    .next();
+                    final WorkflowInstanceTab.Row row = projectPage
+                            .goToTab(WorkflowInstanceTab.class)
+                            .instances()
+                            .iterator()
+                            .next();
+
+                    assertThat(row.isSuccess()).isTrue();
+                    assertThat(row.executionTime()).isEqualTo(1);
+                });
 
-            assertThat(row.isSuccess()).isTrue();
-            assertThat(row.executionTime()).isEqualTo(1);
-        });
     }
+
 }