Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jenkins build.xml files size #729

Closed
joaocgreis opened this issue May 19, 2017 · 7 comments
Closed

Jenkins build.xml files size #729

joaocgreis opened this issue May 19, 2017 · 7 comments

Comments

@joaocgreis
Copy link
Member

According to https://issues.jenkins-ci.org/browse/JENKINS-19022 , the Jenkins Git plugin stores information on build.xml that seems pointless in our use case. A warning is print in every run, pointing to a page with a workaround:

JENKINS-19022: warning: possible memory leak due to Git plugin usage; see: https://wiki.jenkins-ci.org/display/JENKINS/Remove+Git+Plugin+BuildsByBranch+BuildData

There was at least one PR to address this that did not land (jenkinsci/git-plugin#163) and another one that was reverted (jenkinsci/git-plugin#312).

Implementing the workaround is not trivial in our case because of security issues and we would risk breaking the current instance. At least, we should keep track of the situation upstream.

@jeevandongre
Copy link

Is this issue resolve? I am facing the same issue. @joaocgreis

@gibfahn
Copy link
Member

gibfahn commented Jun 20, 2017

Is this issue resolved?

You'd need to raise that with the Jenkins Git Plugin, in https://github.com/jenkinsci/git-plugin/.

@joaocgreis
Copy link
Member Author

@jeevandongre Not on our end, and not upstream as far as I know. If you experiment with it, it would be great if you could share what works and what doesn't, might help us move it forward here. Thanks!

@maclover7
Copy link
Contributor

ping -- is this still warning happening?

@joaocgreis
Copy link
Member Author

Yes. For reference, the build.xml for https://ci.nodejs.org/view/All/job/node-test-pull-request/11314/ is 17M, but becomes only 316K if the BuildData is blindly removed. The main challenge here is testing some solution without risking breaking everything.

@joaocgreis
Copy link
Member Author

Changed the node-compile-windows and node-test-binary-windows jobs to stop using the Jenkins Git plugin and use git commands directly.

Looking quite good for now!
root@infra-digitalocean-ubuntu14-x64-1:/var/lib/jenkins/jobs# find node-test-binary-windows -name build.xml -exec du -sh '{}' \; | grep 19955 | sort -k2
20M     node-test-binary-windows/builds/19955/build.xml
5.1M    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017/axis-RUNNER/win10/axis-RUN_SUBSET/0/builds/19955/build.xml
5.1M    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017/axis-RUNNER/win10/axis-RUN_SUBSET/1/builds/19955/build.xml
5.1M    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017/axis-RUNNER/win10/axis-RUN_SUBSET/2/builds/19955/build.xml
5.1M    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017/axis-RUNNER/win10/axis-RUN_SUBSET/3/builds/19955/build.xml
5.1M    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017/axis-RUNNER/win2008r2-vs2017/axis-RUN_SUBSET/0/builds/19955/build.xml
5.1M    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017/axis-RUNNER/win2008r2-vs2017/axis-RUN_SUBSET/1/builds/19955/build.xml
5.1M    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017/axis-RUNNER/win2008r2-vs2017/axis-RUN_SUBSET/2/builds/19955/build.xml
5.1M    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017/axis-RUNNER/win2008r2-vs2017/axis-RUN_SUBSET/3/builds/19955/build.xml
5.1M    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017-x86/axis-RUNNER/win2012r2/axis-RUN_SUBSET/0/builds/19955/build.xml
5.1M    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017-x86/axis-RUNNER/win2012r2/axis-RUN_SUBSET/1/builds/19955/build.xml
5.1M    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017-x86/axis-RUNNER/win2012r2/axis-RUN_SUBSET/2/builds/19955/build.xml
5.1M    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017-x86/axis-RUNNER/win2012r2/axis-RUN_SUBSET/3/builds/19955/build.xml
root@infra-digitalocean-ubuntu14-x64-1:/var/lib/jenkins/jobs# find node-test-binary-windows -name build.xml -exec du -sh '{}' \; | grep 19956 | sort -k2
12K     node-test-binary-windows/builds/19956/build.xml
4.0K    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017/axis-RUNNER/win10/axis-RUN_SUBSET/0/builds/19956/build.xml
4.0K    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017/axis-RUNNER/win10/axis-RUN_SUBSET/1/builds/19956/build.xml
4.0K    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017/axis-RUNNER/win10/axis-RUN_SUBSET/2/builds/19956/build.xml
4.0K    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017/axis-RUNNER/win10/axis-RUN_SUBSET/3/builds/19956/build.xml
4.0K    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017/axis-RUNNER/win2008r2-vs2017/axis-RUN_SUBSET/0/builds/19956/build.xml
4.0K    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017/axis-RUNNER/win2008r2-vs2017/axis-RUN_SUBSET/1/builds/19956/build.xml
4.0K    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017/axis-RUNNER/win2008r2-vs2017/axis-RUN_SUBSET/2/builds/19956/build.xml
4.0K    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017/axis-RUNNER/win2008r2-vs2017/axis-RUN_SUBSET/3/builds/19956/build.xml
4.0K    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017-x86/axis-RUNNER/win2012r2/axis-RUN_SUBSET/0/builds/19956/build.xml
4.0K    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017-x86/axis-RUNNER/win2012r2/axis-RUN_SUBSET/1/builds/19956/build.xml
4.0K    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017-x86/axis-RUNNER/win2012r2/axis-RUN_SUBSET/2/builds/19956/build.xml
4.0K    node-test-binary-windows/configurations/axis-COMPILED_BY/vs2017-x86/axis-RUNNER/win2012r2/axis-RUN_SUBSET/3/builds/19956/build.xml

For reference, the main things to look at when changing from plugin to script are the refspec and the branch to checkout. If we use this for jobs that don't fetch from the temporary repo, we'll have to look at keys as well.

Windows batch script
TASKKILL /T /F /FI "IMAGENAME eq node.exe"
TASKKILL /T /F /FI "IMAGENAME eq cctest.exe"
TASKKILL /T /F /FI "IMAGENAME eq run-tests.exe"
TASKKILL /T /F /FI "IMAGENAME eq msbuild.exe"
TASKKILL /T /F /FI "IMAGENAME eq mspdbsrv.exe"
TASKKILL /T /F /FI "IMAGENAME eq yes.exe"

IF EXIST .git/index.lock rm -rf .git

SET "ORIGIN_REFS=+refs/heads/master:refs/remotes/origin/master"
SET "ORIGIN_REFS=%ORIGIN_REFS% +refs/heads/v6.x-staging:refs/remotes/origin/v6.x-staging"
SET "ORIGIN_REFS=%ORIGIN_REFS% +refs/heads/v8.x-staging:refs/remotes/origin/v8.x-staging"
SET "ORIGIN_REFS=%ORIGIN_REFS% +refs/heads/v10.x-staging:refs/remotes/origin/v10.x-staging"
SET "ORIGIN_REFS=%ORIGIN_REFS% +refs/heads/v11.x-staging:refs/remotes/origin/v11.x-staging"

@REM Manual git
git --version
git init
git clean -fdx

git fetch --no-tags https://github.com/nodejs/node.git %ORIGIN_REFS%
if %errorlevel% neq 0 echo Problem fetching the main repo.

grep -q ^%TEMP_REPO_SERVER% %USERPROFILE%\.ssh\known_hosts || (ssh-keyscan -t rsa %TEMP_REPO_SERVER% >> %USERPROFILE%\.ssh\known_hosts)
echo off
FOR /F "delims=" %%F IN ('cygpath -u %JENKINS_TMP_KEY%') DO SET "GIT_SSH_COMMAND=ssh -i %%F"
echo on
git fetch --no-tags %TEMP_REPO% +refs/heads/%TEMP_BRANCH%*:refs/remotes/jenkins_tmp/%TEMP_BRANCH%*
if %errorlevel% neq 0 exit /b %errorlevel%
set "GIT_SSH_COMMAND="

git checkout -f jenkins_tmp/%TEMP_BRANCH%
if %errorlevel% neq 0 exit /b %errorlevel%
git reset --hard
if %errorlevel% neq 0 exit /b %errorlevel%
git clean -fdx
if %errorlevel% neq 0 exit /b %errorlevel%

Shell script (not tested)
#!bash -ex

(if [ -e .git/index.lock ]; then rm -rf .git; fi) || true
(if [ $(du -s .git | cut -f1) -gt 5000000 ]; then rm -rf .git; fi) || true

ORIGIN_REFS="+refs/heads/master:refs/remotes/origin/master"
ORIGIN_REFS="$ORIGIN_REFS +refs/heads/v6.x-staging:refs/remotes/origin/v6.x-staging"
ORIGIN_REFS="$ORIGIN_REFS +refs/heads/v8.x-staging:refs/remotes/origin/v8.x-staging"
ORIGIN_REFS="$ORIGIN_REFS +refs/heads/v10.x-staging:refs/remotes/origin/v10.x-staging"
ORIGIN_REFS="$ORIGIN_REFS +refs/heads/v11.x-staging:refs/remotes/origin/v11.x-staging"

# Manual git
git --version
git init
time git clean -fdx || true

time git fetch --no-tags https://github.com/nodejs/node.git $ORIGIN_REFS || echo "Problem fetching the main repo."

grep -q ^$TEMP_REPO_SERVER ~/.ssh/known_hosts || (mkdir -p ~/.ssh && ssh-keyscan -t rsa $TEMP_REPO_SERVER >> ~/.ssh/known_hosts)
time ssh-agent sh -c "ssh-add $JENKINS_TMP_KEY && git fetch --no-tags $TEMP_REPO +refs/heads/${TEMP_BRANCH}*:refs/remotes/jenkins_tmp/${TEMP_BRANCH}*"
rm -f $JENKINS_TMP_KEY

time git checkout -f refs/remotes/jenkins_tmp/$TEMP_BRANCH
time git reset --hard
time git clean -fdx

@joaocgreis
Copy link
Member Author

The solution above fixed the issue for us. Essentially, stop using the Git plugin in the affected jobs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants