Issue with --daemon switch #12
Comments
|
This looks pretty similar than this: #11 (comment) |
|
Yes - these might be duplicate. I noticed this using Buildship for Eclipse which is the gradle plugin from Gradle themselves. It uses the --daemon and nothing we can do about that unfortunately. I am not sure if this is because of the multi project or not, but I have noticed different behavior between the two ways of running the build. This is what I have noticed: This creates the build folder in the root of the master project master-project gradle jfxNative This creates the build folder in the root of the master project master-project So it appears with the daemon flag, the folder structures are not being created correctly - and perhaps that is why the copy is not working? |
|
This is a shot in the dark - but could it be possible that the --daemon doesn't run from the same directory and when it attempts to write the file, it can't locate the relative build directory? Looking on stack overflow: http://stackoverflow.com/questions/14263725/java-files-write-nosuchfileexception asks why it throws this error on a write. This error is thrown when the directory isn't there. http://stackoverflow.com/questions/27335005/java-nio-files-createfile-fails-with-nosuchfileexception Could you try using an absolute path in the copy, rather than the relative path? Perhaps that would address the issue when the --daemon option is used? Thanks. |
|
I noticed that when I ran with daemon this would fail: However, if I changed it to: It worked - so I am thinking that using absolute path at line 123 & 153 for the copy folder might solve the issue. |
|
Thank for your effort in debugging this yourself, maybe that will be solved by fixing issue #13 |
|
So I played with this a little more - created a simple java class to copy this file: public static void main(String[] args) { This throws the following error: If I change it to: it works just fine. Therefore I think the issue is definitely pathing. Thanks for your help! |
|
YES !! Found it! thats the thing I wasn't aware of ! This will be changed with the bugfix for #13 i guess, but I will hold this open until I know this is really fixed. The main-problem is the folder-structure and the "current"-path |
|
Sweet! Thank you again. |
|
@purringpigeon Could you check the current behaviour while using a local build SNAPSHOT-version? |
|
I am not sure how to run this locally? I would be more than happy to, but I haven't done something like that before. I tried changing the version to version: '1.0-SNAPSHOT' but that didn't work. |
|
@purringpigeon I uploaded a new SNAPSHOT-Version: buildscript {
repositories {
maven { url "https://oss.sonatype.org/content/repositories/snapshots" }
}
dependencies {
classpath group: 'de.dynamicfiles.projects.gradle.plugins', name: 'javafx-gradle-plugin', version: '1.1-SNAPSHOT'
}
} |
|
This isn't working anymore. I don't get any errors but I am not getting the project setup or the exe installer. I used to get the jfx set up and app under the: Now when I run it (either with --daemon or not) I get the following: I am missing the rest of the structure and files. Nothing else created. I get no errors now. It completes the build, just doesn't do everything. I am using :gradle jfxNative --daemon or gradle jfxNative |
|
Do you still have the fixed |
That was unwanted, the structure should be now: I uploaded a new SNAPSHOT-version (still being 1.1-SNAPSHOT), so you shouldn't have that wrong created folder below the root-project (I missed some File-constructor). |
|
I think that fixed all the issues.... master-project/app/build/jfx/* I am getting the build working with both --daemon and without --daemon I am reinstalling the gradle plugin in my eclipse and will test that out. But I think that this is working. Here is my build.gradle: buildscript {
dependencies {
//classpath group: 'de.dynamicfiles.projects.gradle.plugins', name: 'javafx-gradle-plugin', version: '1.0-SNAPSHOT'
classpath group: 'de.dynamicfiles.projects.gradle.plugins', name: 'javafx-gradle-plugin', version: '1.1-SNAPSHOT'
}
repositories {
//mavenCentral()
maven { url "https://oss.sonatype.org/content/repositories/snapshots" }
}
}
task setBuildNumber << {
String contents = new File(new File("$projectDir"), 'src/main/deploy/package/windows/appname.template' ).getText( 'UTF-8' )
contents = contents.replaceAll( 'BUILDNUMBER', version )
new File(new File("$projectDir"),'src/main/deploy/package/windows/appname.iss' ).write( contents, 'UTF-8' )
}
task copyRequiredRuntimeConfiguration(type: Sync) {
from 'properties'
into 'build/additionalResources/properties'
include '**/*'
}
apply plugin: 'java'
dependencies{
compile project(':javafx-framework')
compile project(':javafx-project1')
compile project(':javafx-project2')
compile project(':javafx-project3')
compile project(':javafx-project4')
compile project(':javafx-project5')
copyRequiredRuntimeConfiguration
}
apply plugin: 'javafx-gradle-plugin'
copyRequiredRuntimeConfiguration.dependsOn(setBuildNumber)
jfxJar.dependsOn(tasks.copyRequiredRuntimeConfiguration)
jfx {
//Since the path is within this project - need to provide absolute path to resolve it at build time.
additionalAppResources = 'build/additionalResources'
verbose = true
mainClass = 'org.idahosif.application.appname'
appName = 'appname'
manifestAttributes = [
version: "Build 1.0"
]
jfxMainAppJarName = 'appname.jar'
vendor = "My Company"
}HOWEVER The only way to clean the project is to run gradle --stop I don't think we can do that from IDE plug ins. |
|
@purringpigeon so you are still getting the output to the wrong folder The resources for bundling to native bundles and the outcome of the java(fx)packager should result below that dedicated app-bundle-project. Can you confirm this is now where it should be? |
|
No I am getting the correct folders now. master-project So that is good. I am getting the correct structures and the installer. The only issue I have is that clean doesn't work when the daemon is running. |
|
Okay, in that case I would suggest to close this issue and #13 too, what do you think about this? The clean-issue might not have something to do with this gradle-plugin. |
|
I think that this is an issue with the gradle-plug in actually. the clean worked before you made these changes. The clean also works when the --daemon isn't used. It only affects the jfx folders, not the build folders in the other projects. |
|
let me tinker some dummy-multi-project buildscript, hopefully I can recreate that issue myself |
|
I can reproduce that issue now, seems that there is some handler open when running I will look into the JDK, there is some magic regarding version-detection .... |
|
It smells like an issue from the JDK itself:
jfx {
verbose = true
mainClass = 'org.company.application.appname'
appName = 'appname'
manifestAttributes = [
version: "Build 1.0"
]
jfxMainAppJarName = 'appname.jar'
vendor = "My Company"
bundler = "windows.app"
bundleArguments = [
runtime: null
]
}I'll take a deeper look into the JDK .... again |
|
It wouldn't be a big deal for me, but ALL IDE's use the --daemon to work with gradle unfortunately. Glad you found what is going on, perhaps you'll figure out what will fix it. |
|
Just another case of "it's the JDK", I think I've found the spot where the file descriptor-leak exists: As this stream is a DirectoryStream (as documented), this should have to be closed, mostly by using Will try to report this issue, but I will release version 1.1 in the next minutes. |
|
For other visitors regarding Netbeans and Gradle: kelemen/netbeans-gradle-project#269
|
|
@KorvinGump Hi, just a short note: when you are setting your build-file to NOT bundle some JRE, it is possible to call jfx{
// ...
bundleArguments = [
// dont bundle JRE, by setting NULL (it is not recommended, but makes it callable with daemon-mode)
runtime: null
]
} |
|
Current version (8.5.1) makes it possible to ignore the skipping-workaround. |
|
Please note that starting with Gradle 3.0, the daemon is enabled by default, not only for IDEs. |
|
@eskatos thanks for the update to gradle 3 Unfortunately this is not possible, I tried it with different ways, but I would need to "kill" some filehandle held open by the JVM. If someone knows a way, I would be happy to implement that kind of workaround. I've already created some pull-requests on the teamfx-repository, to get more attention about this, but it seems not as important... Current workaround is to manually kill the daemon, or just exclude the bundled JRE, implemented workaround (which even can be disabled too) just does not generate on windows ... not much more I can do here :( |
|
@FibreFoX so one way to work around the issue would be to fork a process so that the leaked file handle does not disturb the Gradle daemon. I don't know the implementation details of this plugin, this may be a big chunk of work, or not. Another way would be to go native and use Win32 CloseHandle function to force-close the file since the handle belong to our process. |
|
@eskatos this plugin uses the internal API of the javapackager, so no process itself is started anyway, which means no "process-fork" will get me to a nice result. I'm not creating the XML-file used by the javapackager as some "converter", this plugin uses the same API which is used by the javapackager itself. As this is workaroundable by using no bundled JRE inside your IDE, and having JRE bundle on the build-system, any native workaround is way too much ... going "native" and use some closehandle ... might work ;) but it is messing too much with the system. As this is some bug in the JDK anyway, I hesitate to do any native action here. Do you have any good source for force-closing filehandles inside java? |
The thing is, with Gradle 3 and the daemon enabled by default, users will have to explicitly disable the daemon to get things working, which is not so nice.
Not really. What about forking a JVM that do exactly what you do already, use the same API used by javapackager? You would have to pass some model to it somehow... This situation is definitely not comfy. Let's hope this will be fixed in a JDK 8 update, not only on 9. It doesn't looks like so for now though ... https://bugs.openjdk.java.net/browse/JDK-8148717 |
|
@eskatos When Gradle 3 is there, maybe the chances are good for manually killing the daemons, the open file-handle created inside JVM is held by the gradle-daemon, killing that daemon might be a valid option ... but as I discussed on the netbeans-gradle-plugin, there is no nice way to manage these daemons. I really don't want to create any native JNA-stuff ;) as it is nasty and doesn't feel right (but I am interestend to find some solution using that stuff, never did any hack like that before, which makes me interested even more into doing that stuff xD). Forking is NO option, as I rely on the loaded classpath by gradle (because of the |
|
Yeah, I knew my suggestion to fork wouldn't please you :) |
|
@eskatos thanks a lot for more suggestions to workaround this bug ;) but I would feel better when the bug would get fixed inside the JDK itself. As this is only problematic on windows-systems (as on linux/macosx there are no locking-issues), this might be a minor issue, but it's a nasty one. As most jenkins-systems run on linux, this "only" hurts local development on windows-systems, where gradle-daemons are present, and not build-systems. Calling The more generic workaround is a manual |
To sum up !What's the problem: All current options:
Possible solutions (on windows only) :
|
|
Just wanted to inform all visitors/watchers: I'm working on replacing the buggy method, first tries using ASM did not work out as expected, using BCEL now, but might still take some time. 😺 This is being worked on! |
|
I think, you have opted-out bundling on Linux/Mac as well? I get the following message on both: Gradle is in daemon-mode, skipped executing bundler, because this would result in some error on clean-task. (JDK-8148717) |
|
@Burtan you can disable that check, by setting Yes, i disabled it by mistake on all platforms :( and with new gradle 3 behaviour of having that daemon by default, this will be more PITA than before. Currently I'm working on some other things, but I hope to provide a working fix soon™️ |
|
Gradle's daemon mode is pretty much mandatory, Gradle's startup can take quite a while and daemon mode fixes that. Assuming that CI environments do not use daemon mode is, at best, a heuristic. I have another stop-gap proposal: Restore the |
|
@toolforger I can't just "restore" the This is no normal memory-leak, it is a file-handle lock! Welcome to the world of file-locking on windows-systems. The tooling provided by the JDK does NOT have permanent JVMs in mind, they think of process-termination after the bundling-process, this is a break-up with the paradigm introduced by gradle. Personally I don't think the daemon mode does that much increase speedup, especially creating bundles should not be seen as "incremental normal development tasks", they are generating some output which has to be executed on the targeted systems after that normal development-phase. Especially regarding the CI-environment: this is even proposed by the gradle-documentations, that CI-envs should not use daemon-mode. |
|
Well, somebody is suppressing the Is this really a file locking scenario? WRT the speedup: Yes it is substantial. Even my minimal JavaFX test-drive project requires 8 seconds to build the model, and the times used to be much worse though I can't say how much of that is due to project size (those were life-size projects, not the toy I'm testing with right now) and much is because optimizations hadn't happened yet. |
|
@toolforger the
To avoid false-positives (and to make CLEAN work), the current implementation inside my plugin simply does not work when daemon-mode is detected, because they are NOT target of the GC.
Sorry to sound rude, but I don't care about this. Creating a DISTRIBUTION BUNDLE should not be part of the normal development-cycle, calling If you find something new I haven't mentioned inside #12 (comment) please don't hesitate to propose some pull-request. |
|
To all watchers: I might have finally beaten this DAEMON to the ground, and now can eat 🍰 Can you please retry using latest Please make sure to have this inside your buildscript for using latest SNAPSHOT-versions: buildscript {
dependencies {
classpath group: 'de.dynamicfiles.projects.gradle.plugins', name: 'javafx-gradle-plugin', version: '8.5.3-SNAPSHOT'
}
repositories {
mavenCentral()
maven { url "https://oss.sonatype.org/content/repositories/snapshots" }
}
}This only works when using gradle daemon mode (which is default on gradle 3), and running on windows using some JDK above and including 1.8.0_60, but ONLY when NOT having This version includes the adjustments required to fix #30 too. (If working, I will push the required changes to github) |
|
@purringpigeon can you recheck this? |
|
The fixes provided for this will be released soon. |
|
I'm very happy to finally close this bug! Thanks for all people who supported me on this one ! |
|
Awesome! 👍 |
I am not sure if this is an issue with the plug in - or gradle. However when I run like this:
gradle jfxNative
Everything works like a champ!
when I use it like this:
gradle jfxNative --daemon
It throws errors when attempting to copy the files to the lib folder:
Couldn't copy dependency commons-codec:commons-codec:1.6
java.nio.file.NoSuchFileException: C:\Users\user.m2\repository\commons-codec\commons-codec\1.6\commons-codec-1.6.jar -> build\jfx\app\lib\commons-codec-1.6.jar
at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
at sun.nio.fs.WindowsFileCopy.copy(WindowsFileCopy.java:205)
at sun.nio.fs.WindowsFileSystemProvider.copy(WindowsFileSystemProvider.java:278)
at java.nio.file.Files.copy(Files.java:1274)
at de.dynamicfiles.projects.gradle.plugins.javafx.tasks.JfxJarTask.lambda$null$1(JfxJarTask.java:123)
at java.lang.Iterable.forEach(Iterable.java:75)
at de.dynamicfiles.projects.gradle.plugins.javafx.tasks.JfxJarTask.lambda$jfxjar$2(JfxJarTask.java:119)
at java.lang.Iterable.forEach(Iterable.java:75)
at de.dynamicfiles.projects.gradle.plugins.javafx.tasks.JfxJarTask.jfxjar(JfxJarTask.java:117)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:75)
at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.doExecute(AnnotationProcessingTaskFactory.java:227)
at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:220)
at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:209)
etc....
Wondering if this a problem with Gradle, or something that needs to be addressed here? So I am posting it. Thanks!
It fails on every jar dependency.
(I have also posted on gradle).
The text was updated successfully, but these errors were encountered: