Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(Docker.*?Plugin): restore the sbt-standard task concurrency tags #1352

Merged
merged 1 commit into from
Jul 8, 2020

Conversation

cchepelov
Copy link
Contributor

@cchepelov cchepelov commented Jul 7, 2020

The purpose of this PR is to restore (and adjust) the concurrency tags offered by sbt (https://www.scala-sbt.org/1.x/docs/Parallel-Execution.html) in order to control parallel execution.

By default, sbt tags the "publishLocal" and "publish" tasks with the Publish and Network tags so that one might limit the number of concurrent tasks competing for resources.

On a large multi-project, multiple subprojects might be attempting to publishLocal simultaneously (i.e. spin up containers in order to build images) which can exhaust memory available to Docker or cause a longer time to build due to contention on the disk. This PR restores tags on the Docker/publish and Docker/publishLocal tasks so that the user can limit publications and/or image constructions to levels consistent with capacity.

The difference with sbt defaults is that the publishLocal task is tagged with the Publish and Disk tags, while the publish task gets the sbt defaults Publish and Network. In practice, the publish task will remain throttled by the availability of the Disk tag as well, as it depends on publishLocal anyway.

"restore"-ish: sbt defines Publish and Network on both publish & publishLocal, we replace Network by Disk on publishLocal to reflect the DockerPlugin's actual resource usage
@cchepelov cchepelov force-pushed the restore_publish_tags branch from 3341a7d to 3f6d4ea Compare July 8, 2020 06:48
@muuki88 muuki88 added the docker label Jul 8, 2020
Copy link
Contributor

@muuki88 muuki88 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this improvement. This makes a lot sense. I have no experience with such large project, building multiple docker images, but I know from even running tests in parallel a smaller jenkins node can die very slowly 😬

Will release ASAP.

@muuki88 muuki88 merged commit 631f809 into sbt:master Jul 8, 2020
@muuki88
Copy link
Contributor

muuki88 commented Jul 8, 2020

Release 1.7.4 is on its way

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants