Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

release 4.23.4 of the build-info-extractor-gradle plugin is slow and uses a lot of memory #487

Closed
mernst-github opened this issue May 2, 2021 · 2 comments · Fixed by #498
Labels
bug Create a report to help us improve

Comments

@mernst-github
Copy link

Describe the bug
We use Gradle 5.6.4 with the build-info-extractor-gradle and gradle-dependency-lock. The latest release (4.23.4) seems to have massively increased the memory usage and resolves dependencies much slower.

  1. The build is much slower. Taking thread dumps of the Gradle daemon reveals this as a culprit:
	at java.lang.RuntimeException.<init>(RuntimeException.java:62)
	at groovy.lang.GroovyRuntimeException.<init>(GroovyRuntimeException.java:37)
	at groovy.lang.MetaClassImpl.invokeConstructor(MetaClassImpl.java:1749)
	at groovy.lang.MetaClassImpl.invokeConstructor(MetaClassImpl.java:1556)
	at org.codehaus.groovy.runtime.InvokerHelper.invokeConstructorOf(InvokerHelper.java:1042)
	at org.codehaus.groovy.runtime.DefaultGroovyMethods.asType(DefaultGroovyMethods.java:11672)
	at org.codehaus.groovy.runtime.dgm$55.doMethodInvoke(Unknown Source)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1217)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1041)
	at org.codehaus.groovy.runtime.InvokerHelper.invokePojoMethod(InvokerHelper.java:999)
	at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:990)
	at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:182)
	at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.asType(ScriptBytecodeAdapter.java:605)
	at org.jfrog.gradle.plugin.artifactory.extractor.listener.ArtifactoryDependencyResolutionListener.updateDependencyMap(ArtifactoryDependencyResolutionListener.groovy:64)

i.e. using 'as String[]' on a List seems to be a Grovvy performance trap.

  1. memory usage, our Gradle daemon started to run out of memory.

With latest version:
image
In comparison 4.21.0:
image

Both while running 'gradle generateLock'.

Versions

  • Extractor version: bad: 4.23.4, good: 4.21.0
  • Operating system:
  • Artifactory Version:

Additional context
A quick inspection of a heap dump seems to indicate massive duplication of version strings. Using G1 collector and -XX:+UseStringDeduplication helps a bit with the heap usage.

@mernst-github mernst-github added the bug Create a report to help us improve label May 2, 2021
@RobiNino
Copy link
Contributor

Hi @mernst-github ,
Thanks for reporting this issue.
We just released version 4.24.2 of the extractor that includes an improvement to the dependency collection mechanism.
The heavy memory usage was due to redundant duplications that were now removed.

We'd appreciate your feedback for the fix.
Thanks

@yahavi yahavi linked a pull request May 28, 2021 that will close this issue
1 task
@mernst-github
Copy link
Author

We've been running 4.24.2 for a few days now with no adverse affects. Thanks for the fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Create a report to help us improve
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants