Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(jenkins): support windows build #1393

Merged
merged 129 commits into from
Mar 3, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
129 commits
Select commit Hold shift + click to select a range
816c0a4
ci(jenkins): support windows build
v1v Sep 25, 2019
2fc0dfd
chore: for testing purposes
v1v Sep 25, 2019
ada9d64
ci(jenkins): support java and minor changes
v1v Sep 25, 2019
30bc58d
chore: let's print the env
v1v Sep 25, 2019
570dc04
ci(jenkins): bump version with jdk
v1v Sep 25, 2019
fed751e
ci(jenkins): unset JAVA_HOME to use the bundle
v1v Sep 25, 2019
252e8d7
chore: debug
v1v Sep 25, 2019
1dd9661
ci(jenkins): install nodejs with choco
v1v Sep 26, 2019
becd3de
ci(jenkins): remove support to appveyor
v1v Sep 26, 2019
13ae974
ci(jenkins): use docker instead
v1v Sep 26, 2019
203aa17
chore: fix env variable call
v1v Sep 26, 2019
1ddbb6a
chore: more debug details
v1v Sep 26, 2019
d558da3
ci(jenkins): install docker-compose
v1v Sep 26, 2019
ddae994
Merge branch 'master' into feature/windows-build
v1v Sep 30, 2019
60e94e7
Revert "ci(jenkins): remove support to appveyor"
v1v Sep 30, 2019
426db13
ci(jenkins): docker is not supported
v1v Sep 30, 2019
c3ce5d0
Merge branch 'feature/windows-build' of github.com:v1v/apm-agent-node…
v1v Sep 30, 2019
efdb9cd
ci(jenkins): install dependent modules
v1v Sep 30, 2019
8160e9a
ci(jenkins): install elasticsearch
v1v Sep 30, 2019
9563031
ci(jenkins): running independently
v1v Sep 30, 2019
42d8c2d
ci(jenkins): without the infra
v1v Sep 30, 2019
2455329
ci(jenkins): support more nodejs versions
v1v Sep 30, 2019
49ac3c6
ci(jenkins): env list not required
v1v Sep 30, 2019
adcaafd
ci(jenkins): debugging purposes
v1v Sep 30, 2019
135752f
ci(jenkins): jenkins is now the place
v1v Sep 30, 2019
859e5ac
Revert "chore: for testing purposes"
v1v Sep 30, 2019
279ed7d
ci(jenkins): workflow
v1v Sep 30, 2019
2920950
ci(jenkins): disable performance
v1v Sep 30, 2019
46545c3
Revert "ci(jenkins): disable performance"
v1v Sep 30, 2019
d97b21d
test: fix windows process.title
Qard Oct 2, 2019
5991e4f
ci(config): do not use symlink
Qard Oct 3, 2019
88154f7
Revert "ci(jenkins): without the infra"
v1v Oct 4, 2019
b08826d
ci(jenkins): install java
v1v Oct 4, 2019
92966f2
chore: enable cassandra
v1v Oct 4, 2019
0c8a798
ci(jenkins): for testing purposes
v1v Oct 4, 2019
d2547c8
ci(jenkins): force the installation
v1v Oct 4, 2019
f379c62
ci(jenkins): hardcode the string
v1v Oct 4, 2019
7ebcf97
ci(jenkins): skip reboot
v1v Oct 4, 2019
d94f694
ci(jenkins): uninstall current jdk
v1v Oct 4, 2019
11ff86f
Revert "ci(jenkins): for testing purposes"
v1v Oct 4, 2019
88e83b1
Merge remote-tracking branch 'upstream/master' into feature/windows-b…
v1v Oct 4, 2019
a82931b
chore: testing purposes
v1v Oct 4, 2019
7079fa3
ci(jenkins): jdk8 get rebooted
v1v Oct 4, 2019
110dfa1
ci(jenkins): avoid windows limit
v1v Oct 4, 2019
3f45610
ci(jenkins): fix download
v1v Oct 4, 2019
95ceac4
ci(jenkins): fix download and path
v1v Oct 4, 2019
05fb1da
ci(jenkins): enable other tools
v1v Oct 4, 2019
cc0bdd0
ci(jenkins): downgrade version
v1v Oct 4, 2019
5f32bbb
ci(jenkins): use choco instead
v1v Oct 4, 2019
5761c41
ci(jenkins): refreshenv
v1v Oct 4, 2019
cf41802
ci(jenkins): use docker instead
v1v Oct 7, 2019
0b4e264
ci(jenkins): only elasticsearch
v1v Oct 7, 2019
a587e88
ci(jenkins): using docker
v1v Oct 7, 2019
25a824e
ci(jenkins): redis with docker
v1v Oct 7, 2019
1d8a6ef
ci(jenkins): cassandra with docker
v1v Oct 7, 2019
46c13d5
ci(jenkins): mongodb with docker
v1v Oct 7, 2019
750eaad
ci(jenkins): docker debug
v1v Oct 7, 2019
9eb28c8
ci(jenkins): postgres docker
v1v Oct 7, 2019
7a15ffc
ci(jenkins): mssql docker
v1v Oct 7, 2019
9cff595
ci(jenkins): run services in linux
v1v Oct 8, 2019
96a3025
ci(jenkins): fix interpolation
v1v Oct 8, 2019
5749a66
ci(jenkins): enable docker-compose wait
v1v Oct 8, 2019
438dc93
ci(jenkins): debug and interpolate
v1v Oct 8, 2019
ed80403
ci(jenkins): enable windows docker
v1v Oct 8, 2019
a55745c
ci(jenkins): not supported yet
v1v Oct 8, 2019
7d1952a
ci(jenkins): mssql docker
v1v Oct 8, 2019
f07a316
ci(jenkins): mysql docker
v1v Oct 8, 2019
11d3d5c
ci(jenkins): docker-compose
v1v Oct 9, 2019
9314a38
ci(jenkins): traditional docker
v1v Oct 9, 2019
99c3f12
ci(jenkins): override SA password
v1v Oct 9, 2019
6be12a3
ci(jenkins): mysql warm up
v1v Oct 9, 2019
1b7744b
ci(jenkins): change mysql warm up
v1v Oct 9, 2019
e54cd4e
ci(jenkins): tweak java memalloc
v1v Oct 9, 2019
d38a6a5
ci(jenkins): bind mysql
v1v Oct 9, 2019
782a76b
ci(jenkins): enable remote mysql access
v1v Oct 10, 2019
aa6b82d
ci(jenkins): mysql user elastic for windows
v1v Oct 10, 2019
44a7628
ci(jenkins): postgres remote access
v1v Oct 10, 2019
d7ddb62
ci(jenkins): linux docker unsupported
v1v Oct 10, 2019
e599c75
ci(jenkins): docker-compose unused
v1v Oct 11, 2019
1992f26
Merge remote-tracking branch 'upstream/master' into feature/windows-b…
v1v Oct 11, 2019
0484211
ci(jenkins): passwords env var
v1v Oct 11, 2019
e42d4b4
Revert "chore: testing purposes"
v1v Oct 11, 2019
8d49a9f
ci(jenkins): gather env details
v1v Oct 11, 2019
23558e6
ci(jenkins): if windows
v1v Oct 11, 2019
af7a215
ci(jenkins): sh before the finally
v1v Oct 11, 2019
94931b6
ci(jenkins): more debug
v1v Oct 11, 2019
ea7c4cf
ci(jenkins): enforce PATH
v1v Oct 11, 2019
0a1d38e
ci(jenkins): use withEnv toplevel
v1v Oct 11, 2019
8bb0493
Revert "ci(jenkins): use withEnv toplevel"
v1v Oct 14, 2019
8a3d00c
Revert "ci(jenkins): enforce PATH"
v1v Oct 14, 2019
4fbcf03
Revert "ci(jenkins): more debug"
v1v Oct 14, 2019
5ab6c83
Revert "ci(jenkins): sh before the finally"
v1v Oct 14, 2019
b032aec
Revert "ci(jenkins): if windows"
v1v Oct 14, 2019
d911932
Revert "ci(jenkins): gather env details"
v1v Oct 14, 2019
da21916
ci(jenkins): withEnv to avoid env clashes
v1v Oct 14, 2019
9c4bf41
Merge branch 'master' into feature/windows-build
v1v Oct 14, 2019
126da47
test(bluebird): properly handle Windows line breaks
Qard Oct 19, 2019
86210cc
test(pg): set fallback when PGUSER is absent
Qard Oct 19, 2019
fae14b5
Merge remote-tracking branch 'upstream/master' into feature/windows-b…
v1v Dec 15, 2019
731f144
ci(jenkins): run docker services in linux
v1v Dec 15, 2019
01c747c
ci(jenkins): add verbose for the install in windows
v1v Dec 15, 2019
8e8bbe1
ci(jenkins): debug the ps1
v1v Dec 15, 2019
53cec38
ci(jenkins): some leftovers
v1v Dec 15, 2019
3de38cc
ci(jenkins): run quietly and detached
v1v Dec 15, 2019
dd2a260
ci(jenkins): redirect logs to a file
v1v Dec 15, 2019
dcb584c
ci(jenkins): otherwise uses nodejs.commandline
v1v Dec 15, 2019
8ba3d6e
ci(jenkins): run docker services in linux
v1v Dec 15, 2019
225b296
ci(jenkins): some leftovers
v1v Dec 15, 2019
1608f88
ci(jenkins): run quietly and detached
v1v Dec 15, 2019
6e8ba8b
ci(jenkins): redirect logs to a file
v1v Dec 15, 2019
d427e3c
Merge branch 'feature/windows-build-with-linux-service' of github.com…
v1v Dec 15, 2019
18084e9
ci(jenkins): default mssql password
v1v Dec 15, 2019
859caae
ci(jenkins): default mysql user/password
v1v Dec 15, 2019
c21dc48
ci(jenkins): docker for windows unrequired
v1v Dec 15, 2019
b628ce0
ci(jenkins): windows test output
v1v Dec 15, 2019
941dacc
Merge pull request #1 from v1v/feature/windows-build-with-linux-service
v1v Dec 15, 2019
46f3c08
Merge branch 'master' into feature/windows-build
v1v Dec 15, 2019
f9953fc
ci(jenkins): junit in windows not yet enabled
v1v Dec 15, 2019
4a6650e
ci(jenkins): multiple node versions
v1v Dec 15, 2019
0b652ce
Revert "ci(jenkins): multiple node versions"
v1v Dec 15, 2019
bdb5890
Revert "ci(jenkins): remove support to appveyor"
v1v Dec 15, 2019
324c618
ci(jenkins): reset scripts to master version
v1v Dec 15, 2019
03923bf
ci(jenkins): appveyor is not required
v1v Dec 15, 2019
f84bb32
Merge branch 'master' into feature/windows-build
v1v Dec 26, 2019
6d7d7b0
Merge branch 'master' into feature/windows-build
v1v Jan 7, 2020
f524379
Merge remote-tracking branch 'upstream/master' into feature/windows-b…
v1v Mar 3, 2020
891b38f
ci(jenkins): use installTools step
v1v Mar 3, 2020
1b8d9ec
ci(jenkins): store JUnit for the Windows stage
v1v Mar 3, 2020
35e355f
Revert "ci(jenkins): store JUnit for the Windows stage"
v1v Mar 3, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 0 additions & 28 deletions .appveyor.yml

This file was deleted.

85 changes: 66 additions & 19 deletions .ci/Jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -80,8 +80,13 @@ pipeline {
node['NODEJS_VERSION'].each{ version ->
parallelTasks["Node.js-${version}"] = generateStep(version: version)
parallelTasks["Node.js-${version}-async-hooks-false"] = generateStep(version: version, disableAsyncHooks: true)
// TODO: to be enabled if required.
// parallelTasks["Windows-Node.js-${version}"] = generateStepForWindows(version: version)
}

// Only 12 for the time being
parallelTasks["Windows-Node.js-12"] = generateStepForWindows(version: '12')

// PRs don't require to run here as it's now managed within the linting pipeline
if (!env.CHANGE_ID) {
// Linting in parallel with the test stage
Expand Down Expand Up @@ -352,27 +357,25 @@ def generateStep(Map params = [:]){
def disableAsyncHooks = params.get('disableAsyncHooks', false)
return {
node('linux && immutable'){
try {
env.HOME = "${WORKSPACE}"
if (disableAsyncHooks) {
env.ELASTIC_APM_ASYNC_HOOKS = 'false'
}
deleteDir()
unstash 'source'
dir("${BASE_DIR}"){
retry(2){
sleep randomNumber(min:10, max: 30)
sh(label: "Run Tests", script: """.ci/scripts/test.sh "${version}" "${tav}" "${edge}" """)
withEnv(["VERSION=${version}", "ELASTIC_APM_ASYNC_HOOKS=${disableAsyncHooks}"]) {
try {
deleteDir()
unstash 'source'
dir("${BASE_DIR}"){
retry(2){
sleep randomNumber(min:5, max: 10)
sh(label: "Run Tests", script: """.ci/scripts/test.sh "${version}" "${tav}" "${edge}" """)
}
}
} catch(e){
error(e.toString())
} finally {
docker.image('node:12').inside("-v ${WORKSPACE}/${BASE_DIR}:/app"){
sh(label: "Convert Test results to JUnit format", script: 'cd /app && .ci/scripts/convert_tap_to_junit.sh')
}
junit(allowEmptyResults: true, keepLongStdio: true, testResults: "${BASE_DIR}/**/junit-*.xml")
codecov(repo: env.REPO, basedir: "${BASE_DIR}", secret: "${CODECOV_SECRET}")
}
} catch(e){
error(e.toString())
} finally {
docker.image('node:12').inside("-v ${WORKSPACE}/${BASE_DIR}:/app"){
sh(label: "Convert Test results to JUnit format", script: 'cd /app && .ci/scripts/convert_tap_to_junit.sh')
}
junit(allowEmptyResults: true, keepLongStdio: true, testResults: "${BASE_DIR}/**/junit-*.xml")
codecov(repo: env.REPO, basedir: "${BASE_DIR}", secret: "${CODECOV_SECRET}")
}
}
}
Expand Down Expand Up @@ -444,3 +447,47 @@ def getSmartTAVContext() {
}
}
}

def generateStepForWindows(Map params = [:]){
def version = params?.version
def disableAsyncHooks = params.get('disableAsyncHooks', false)
return {
sh label: 'Prepare services', script: ".ci/scripts/windows/prepare-test.sh ${version}"
def linuxIp = sh(label: 'Get IP', script: '''hostname -I | awk '{print $1}' ''', returnStdout: true)?.trim()
node('windows-2019-docker-immutable'){
// When installing with choco the PATH might not be updated within the already connected worker.
withEnv(["PATH=${PATH};C:\\Program Files\\nodejs",
"VERSION=${version}",
"ELASTIC_APM_ASYNC_HOOKS=${disableAsyncHooks}",
"CASSANDRA_HOST=${linuxIp}",
"ES_HOST=${linuxIp}",
"MEMCACHED_HOST=${linuxIp}",
"MONGODB_HOST=${linuxIp}",
"MSSQL_HOST=${linuxIp}",
"MYSQL_HOST=${linuxIp}",
"PGHOST=${linuxIp}",
"REDIS_HOST=${linuxIp}"]) {
try {
deleteDir()
unstash 'source'
dir(BASE_DIR) {
installTools([ [tool: 'nodejs', version: "${version}" ] ])
bat label: 'Tool versions', script: '''
npm --version
node --version
'''
bat 'npm install'
mdelapenya marked this conversation as resolved.
Show resolved Hide resolved
bat 'node test/test.js'
}
} catch(e){
error(e.toString())
} finally {
echo 'JUnit archiving no yet in place'
}
}
}

// If the above execution failed, then it will not reach this section. TBD
sh label: 'Stop services', script: ".ci/scripts/windows/stop-test.sh ${version}"
}
}
16 changes: 16 additions & 0 deletions .ci/scripts/windows/prepare-test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/usr/bin/env bash
set -exo pipefail

NODE_VERSION=${1:?Nodejs version missing NODE_VERSION is not set}

NODE_VERSION=${NODE_VERSION} \
USER_ID="$(id -u):$(id -g)" \
docker-compose \
--no-ansi \
--log-level ERROR \
-f .ci/docker/docker-compose-all.yml \
up \
--build \
--remove-orphans \
--quiet-pull \
--detach
16 changes: 16 additions & 0 deletions .ci/scripts/windows/stop-test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/usr/bin/env bash
set -exo pipefail

NODE_VERSION=${1:?Nodejs version missing NODE_VERSION is not set}

NODE_VERSION=${NODE_VERSION} docker-compose \
--no-ansi \
-f .ci/docker/docker-compose-all.yml \
logs \
--timestamps > docker-compose-logs.txt

NODE_VERSION=${NODE_VERSION} docker-compose \
--no-ansi \
--log-level ERROR \
-f .ci/docker/docker-compose-all.yml \
down -v
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,5 @@ build
coverage
node_modules
test/benchmarks/.tmp
coverage.lcov
watson marked this conversation as resolved.
Show resolved Hide resolved
test-suite-output.tap
watson marked this conversation as resolved.
Show resolved Hide resolved
5 changes: 1 addition & 4 deletions test/agent.js
Original file line number Diff line number Diff line change
Expand Up @@ -1373,10 +1373,7 @@ function assertMetadata (t, payload) {
t.equal(payload.process.pid, process.pid)
t.ok(payload.process.pid > 0, 'should have a pid greater than 0')
t.ok(payload.process.title, 'should have a process title')
t.ok(
/(npm|node)/.test(payload.process.title),
`process.title should contain expected value (was: "${payload.process.title}")`
)
t.equal(payload.process.title, process.title)
t.deepEqual(payload.process.argv, process.argv)
t.ok(payload.process.argv.length >= 2, 'should have at least two process arguments')
}
Expand Down
26 changes: 22 additions & 4 deletions test/config.js
Original file line number Diff line number Diff line change
Expand Up @@ -470,13 +470,22 @@ test('serviceName defaults to package name', function (t) {
{
action: 'mkdirp',
dir: path.join(tmp, 'node_modules')
},
{
}
]

if (process.platform === 'win32') {
files.push({
action: 'npm link',
from: path.resolve(__dirname, '..'),
to: tmp
})
} else {
files.push({
action: 'symlink',
from: path.resolve(__dirname, '..'),
to: path.join(tmp, 'node_modules/elastic-apm-node')
}
]
})
}

// NOTE: Reduce the sequence to a promise chain rather
// than using Promise.all(), as the tasks are dependent.
Expand All @@ -492,6 +501,15 @@ test('serviceName defaults to package name', function (t) {
case 'symlink': {
return symlink(file.from, file.to)
}
case 'npm link': {
return exec('npm link', {
cwd: file.from
}).then(() => {
return exec('npm link elastic-apm-node', {
cwd: file.to
})
})
}
}
})
}, Promise.resolve())
Expand Down
2 changes: 1 addition & 1 deletion test/instrumentation/modules/bluebird/bluebird.js
Original file line number Diff line number Diff line change
Expand Up @@ -927,7 +927,7 @@ test('Promise.promisify', function (t) {

readFile(__filename, 'utf8').then(function (contents) {
var firstLine = contents.split('\n')[0]
t.equal(firstLine, '\'use strict\'')
t.ok(/use strict/.test(firstLine))
t.equal(ins.currentTransaction.id, trans.id)
})
})
Expand Down
10 changes: 8 additions & 2 deletions test/instrumentation/modules/pg/_utils.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,10 @@ exports.reset = reset
exports.loadData = loadData

function reset (cb) {
var client = new Client({ database: 'postgres' })
var client = new Client({
database: 'postgres',
user: process.env.PGUSER || 'postgres'
})

client.connect(function (err) {
if (err) throw err
Expand All @@ -22,7 +25,10 @@ function reset (cb) {
}

function loadData (cb) {
var client = new Client({ database: 'test_elastic_apm' })
var client = new Client({
database: 'test_elastic_apm',
user: process.env.PGUSER || 'postgres'
})

client.connect(function (err) {
if (err) throw err
Expand Down
5 changes: 4 additions & 1 deletion test/instrumentation/modules/pg/knex.js
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,10 @@ function createClient (t, cb) {
setup(function () {
knex = Knex({
client: 'pg',
connection: 'postgres:///test_elastic_apm'
connection: {
database: 'test_elastic_apm',
user: process.env.PGUSER || 'postgres'
}
})
t.on('end', () => {
knex.destroy(function (err) {
Expand Down
11 changes: 8 additions & 3 deletions test/instrumentation/modules/pg/pg.js
Original file line number Diff line number Diff line change
Expand Up @@ -504,7 +504,8 @@ function assertSpan (t, span, sql) {
function createClient (cb) {
setup(function () {
queryable = new pg.Client({
database: 'test_elastic_apm'
database: 'test_elastic_apm',
user: process.env.PGUSER || 'postgres'
})
queryable.connect(function (err) {
if (err) throw err
Expand All @@ -520,11 +521,15 @@ function createPool (cb) {
if (semver.satisfies(pgVersion, '<6.0.0')) {
queryable = pg
connector = function connector (cb) {
return pg.connect('postgres:///test_elastic_apm', cb)
return pg.connect({
database: 'test_elastic_apm',
user: process.env.PGUSER || 'postgres'
}, cb)
}
} else {
var pool = new pg.Pool({
database: 'test_elastic_apm'
database: 'test_elastic_apm',
user: process.env.PGUSER || 'postgres'
})
queryable = pool
connector = function connector (cb) {
Expand Down
21 changes: 0 additions & 21 deletions test/script/appveyor/install-cassandra.ps1

This file was deleted.

27 changes: 0 additions & 27 deletions test/script/appveyor/install-elasticsearch.ps1

This file was deleted.

17 changes: 0 additions & 17 deletions test/script/appveyor/install-redis.ps1

This file was deleted.