Be more granular with SDK and NDK stamp files #37
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
looks ok to me |
| @@ -36,7 +36,7 @@ | |||
| <Target Name="_UnzipFiles" | |||
| DependsOnTargets="_DetermineItems" | |||
| Inputs="@(_PlatformAndroidSdkItem->'$(AndroidToolchainCacheDirectory)\%(Identity)')" | |||
| Outputs="$(AndroidToolchainDirectory)\.stamp-sdk"> | |||
| Outputs="@(_PlatformAndroidSdkItem->'$(AndroidToolchainDirectory)\sdk\%(DestDir)\.stamp-sdk') $(AndroidToolchainDirectory)\ndk\.stamp-ndk"> | |||
There was a problem hiding this comment.
Not everything sets %(AndroidSdkItem.DestDir), and thus %(DestDir) may be empty .
Additionally, it's not necessarily easy to detect what %(DestDir) could be; there could be multiple "top level" directories extracted from a single .zip. (For the most part, our extraction code doesn't care.)
I would thus suggest using %(Identity) within the .stamp filename:
@(_PlatformAndroidSdkItem->'$(AndroidToolchainDirectory)\sdk\.stamp-%(Identity)')
Finally, you need to use ;, not , to separate file paths. Thus:
Outputs="@(_PlatformAndroidSdkItem->'$(AndroidToolchainDirectory)\sdk\.stamp-%(Identity)');$(AndroidToolchainDirectory)\ndk\.stamp-ndk">
There was a problem hiding this comment.
if %(DestDir) is empty then the path will just be one level up, but I see your point and it makes sense.
Previously we used a single stamp file to detect whether SDK/NDK need to be updated/recreate. This worked fine if nobody touched the toolchain directory. However, if any of the directories (sdk, ndk or anything in them) were removed, build would *not* recreate the toolchain *unless* the stamp file in the root folder would be deleted as well. This patch creates a stamp file per component directory for all the NDK and SDK components and thus the content will be restored should the directory be removed. It can be further improved, probably, by not removing the entire directory prior to unzipping files as we do now but instead unpack only those components that are missing.
|
What happens on "sdk component upgrade", e.g. Can you test this scenario, and confirm that it works (i.e. that my vague recollection is wrong)? |
|
Yep, will test. |
|
Replacing older files works fine here. Maybe you meant using File.Move across filesystems, as that's what doesn't work? |
radical
pushed a commit
that referenced
this pull request
May 8, 2018
[generator] Use the *real* Mono.Options.
radical
pushed a commit
that referenced
this pull request
May 8, 2018
When `JniRuntime.CreationOptions.DestroyRuntimeOnDispose` is true,
`JavaVM::DestroyJavaVM()` will be invoked when the `JniRuntime`
instance is disposed *or* finalized.
`JreRuntime.CreateJreVM()` would *always* set
`DestroyRuntimeOnDispose` to true, because it called
`JNI_CreateJavaVM()`, so *of course* you'd want to destroy the
Java VM, right?
Which brings us to unit tests. I don't know of any "before all test
fixtures run" and "after all test fixtures run" extension points,
which means:
1. The JVM needs to be created implicitly, "on demand."
2. There's no good way to destroy the JVM created in (1) after all
tests have finished executing.
Which *really* means that the `JreRuntime` instance is *finalized*,
which sets us up for the unholy trifecta of AppDomain unloads,
finalizers, and JVM shutdown:
For unknown reasons, ~randomly, when running the unit tests (e.g.
`make run-tests`), the test runner will *hang*, indefinitely.
Attaching `lldb` and triggering a backtrace shows the unholy trifecta:
Finalization:
thread #4: tid = 0x403831, 0x00007fff9656bdb6 libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'tid_1403'
...
frame #10: 0x00000001001ccb4a mono64`mono_gc_run_finalize(obj=<unavailable>, data=<unavailable>) + 938 at gc.c:256 [opt]
frame #11: 0x00000001001cdd4a mono64`finalizer_thread [inlined] finalize_domain_objects + 51 at gc.c:681 [opt]
frame #12: 0x00000001001cdd17 mono64`finalizer_thread(unused=<unavailable>) + 295 at gc.c:730 [opt]
JVM destruction:
thread #4: tid = 0x403831, 0x00007fff9656bdb6 libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'tid_1403'
frame #0: 0x00007fff9656bdb6 libsystem_kernel.dylib`__psynch_cvwait + 10
frame #1: 0x00007fffa04d4728 libsystem_pthread.dylib`_pthread_cond_wait + 767
frame #2: 0x000000010ba5bc76 libjvm.dylib`os::PlatformEvent::park() + 192
frame #3: 0x000000010ba38e32 libjvm.dylib`ParkCommon(ParkEvent*, long) + 42
frame #4: 0x000000010ba39708 libjvm.dylib`Monitor::IWait(Thread*, long) + 168
frame #5: 0x000000010ba398f0 libjvm.dylib`Monitor::wait(bool, long, bool) + 246
frame #6: 0x000000010bb3dca2 libjvm.dylib`Threads::destroy_vm() + 80
frame #7: 0x000000010b8fd665 libjvm.dylib`jni_DestroyJavaVM + 254
AppDomain unload:
thread #37: tid = 0x4038fb, 0x00007fff9656bdb6 libsystem_kernel.dylib`__psynch_cvwait + 10
frame #0: 0x00007fff9656bdb6 libsystem_kernel.dylib`__psynch_cvwait + 10
frame #1: 0x00007fffa04d4728 libsystem_pthread.dylib`_pthread_cond_wait + 767
frame #2: 0x0000000100234a7f mono64`mono_os_cond_timedwait [inlined] mono_os_cond_wait(cond=0x0000000102016e50, mutex=0x0000000102016e10) + 11 at mono-os-mutex.h:105 [opt]
frame #3: 0x0000000100234a74 mono64`mono_os_cond_timedwait(cond=0x0000000102016e50, mutex=0x0000000102016e10, timeout_ms=<unavailable>) + 164 at mono-os-mutex.h:120 [opt]
frame #4: 0x0000000100234828 mono64`_wapi_handle_timedwait_signal_handle(handle=0x0000000000000440, timeout=4294967295, alertable=1, poll=<unavailable>, alerted=0x0000700000a286f4) + 536 at handles.c:1554 [opt]
frame #5: 0x0000000100246370 mono64`wapi_WaitForSingleObjectEx(handle=<unavailable>, timeout=<unavailable>, alertable=<unavailable>) + 592 at wait.c:189 [opt]
frame #6: 0x00000001001c832e mono64`mono_domain_try_unload [inlined] guarded_wait(timeout=4294967295, alertable=1) + 30 at appdomain.c:2390 [opt]
frame #7: 0x00000001001c8310 mono64`mono_domain_try_unload(domain=0x000000010127ccb0, exc=0x0000700000a287a0) + 416 at appdomain.c:2482 [opt]
frame #8: 0x00000001001c7db2 mono64`ves_icall_System_AppDomain_InternalUnload [inlined] mono_domain_unload(domain=<unavailable>) + 20 at appdomain.c:2379 [opt]
frame #9: 0x00000001001c7d9e mono64`ves_icall_System_AppDomain_InternalUnload(domain_id=<unavailable>) + 46 at appdomain.c:2039 [opt]
This randomly results in deadlock, and hung Jenkins bots.
Fix this behavior by altering `JreRuntime.CreateJreVM()` to *not*
override the value of
`JniRuntime.CreationOptions.DestroyRuntimeOnDispose`. This allows the
constructor of the `JreRuntime` instance to decide whether or not the
JVM is destroyed.
In the case of TestJVM, we *don't* want to destroy the JVM.
This prevents the JVM from being destroyed, which in turn prevents the
hang during process shutdown.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously we used a single stamp file to detect
whether SDK/NDK need to be updated/recreate. This
worked fine if nobody touched the toolchain directory.
However, if any of the directories (sdk, ndk or anything in them)
were removed, build would not recreate the toolchain unless
the stamp file in the root folder would be deleted as well.
This patch creates a stamp file per component directory for all
the NDK and SDK components and thus the content will be restored
should the directory be removed.
It can be further improved, probably, by not removing the entire
directory prior to unzipping files as we do now but instead unpack
only those components that are missing.