Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dynamic circleci config for streamlining test execution #6561

Merged
merged 11 commits into from
Jun 23, 2021
Merged

dynamic circleci config for streamlining test execution #6561

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
51e51d1
dynamic circleci config.
raulk Jun 22, 2021
ec6c394
remove dynamic circleci.
raulk Jun 22, 2021
539f8c1
fix test that references private symbols; avoid hacky test selection …
raulk Jun 22, 2021
c6c202d
add makefile target.
raulk Jun 22, 2021
b74ad79
fix dangling import.
raulk Jun 22, 2021
da96414
switch test output format.
raulk Jun 22, 2021
211751f
declare some tests as expensive.
raulk Jun 23, 2021
6017715
(test) deliberately check in an outdated .circleci/config.yml to test…
raulk Jun 23, 2021
0c31676
Makefile: add circleci target as dependency of gen target.
raulk Jun 23, 2021
2c605e0
Revert "(test) deliberately check in an outdated .circleci/config.yml…
raulk Jun 23, 2021
f434140
deterministic order for rest unit tests.
raulk Jun 23, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
223 changes: 119 additions & 104 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ jobs:
- run:
command: make debug

test: &test
test:
description: |
Run tests with gotestsum.
parameters: &test-params
Expand All @@ -123,34 +123,28 @@ jobs:
type: string
default: "-timeout 30m"
description: Flags passed to go test.
packages:
target:
type: string
default: "./..."
description: Import paths of packages to be tested.
winpost-test:
type: string
default: "0"
deadline-test:
type: string
default: "0"
proofs-log-test:
type: string
default: "0"
test-suite-name:
suite:
type: string
default: unit
description: Test suite name to report to CircleCI.
gotestsum-format:
type: string
default: pkgname-and-test-fails
default: standard-verbose
description: gotestsum format. https://github.com/gotestyourself/gotestsum#format
coverage:
type: string
default: -coverprofile=coverage.txt -coverpkg=github.com/filecoin-project/lotus/...
description: Coverage flag. Set to the empty string to disable.
codecov-upload:
type: boolean
default: false
default: true
description: |
Upload coverage report to https://codecov.io/. Requires the codecov API token to be
set as an environment variable for private projects.
Expand All @@ -168,26 +162,24 @@ jobs:
- run:
name: go test
environment:
LOTUS_TEST_WINDOW_POST: << parameters.winpost-test >>
LOTUS_TEST_DEADLINE_TOGGLING: << parameters.deadline-test >>
TEST_RUSTPROOFS_LOGS: << parameters.proofs-log-test >>
SKIP_CONFORMANCE: "1"
command: |
mkdir -p /tmp/test-reports/<< parameters.test-suite-name >>
mkdir -p /tmp/test-reports/<< parameters.suite >>
mkdir -p /tmp/test-artifacts
gotestsum \
--format << parameters.gotestsum-format >> \
--junitfile /tmp/test-reports/<< parameters.test-suite-name >>/junit.xml \
--jsonfile /tmp/test-artifacts/<< parameters.test-suite-name >>.json \
--junitfile /tmp/test-reports/<< parameters.suite >>/junit.xml \
--jsonfile /tmp/test-artifacts/<< parameters.suite >>.json \
-- \
<< parameters.coverage >> \
<< parameters.go-test-flags >> \
<< parameters.packages >>
<< parameters.target >>
no_output_timeout: 30m
- store_test_results:
path: /tmp/test-reports
- store_artifacts:
path: /tmp/test-artifacts/<< parameters.test-suite-name >>.json
path: /tmp/test-artifacts/<< parameters.suite >>.json
- when:
condition: << parameters.codecov-upload >>
steps:
Expand All @@ -198,26 +190,6 @@ jobs:
command: |
bash <(curl -s https://codecov.io/bash)

test-chain:
<<: *test
test-node:
<<: *test
test-storage:
<<: *test
test-cli:
<<: *test
test-short:
<<: *test
test-window-post:
<<: *test
test-window-post-dispute:
<<: *test
test-deadline-toggling:
<<: *test
test-terminate:
<<: *test
check-proofs-multicore-sdr:
<<: *test
test-conformance:
description: |
Run tests using a corpus of interoperable test vectors for Filecoin
Expand Down Expand Up @@ -598,7 +570,6 @@ jobs:
name: publish snap
command: snapcraft push *.snap --release << parameters.channel >>


build-and-push-image:
description: build and push docker images to public AWS ECR registry
executor: aws-cli/default
Expand Down Expand Up @@ -776,64 +747,120 @@ workflows:
- gen-check
- docs-check
- test:
codecov-upload: true
test-suite-name: full
- test-chain:
codecov-upload: true
test-suite-name: chain
packages: "./chain/..."
- test-node:
codecov-upload: true
test-suite-name: node
packages: "./node/..."
- test-storage:
codecov-upload: true
test-suite-name: storage
packages: "./storage/... ./extern/..."
- test-cli:
codecov-upload: true
test-suite-name: cli
packages: "./cli/... ./cmd/... ./api/..."
- test-window-post:
codecov-upload: true
go-test-flags: "-run=TestWindowedPost"
winpost-test: "1"
test-suite-name: window-post
- test-window-post-dispute:
codecov-upload: true
go-test-flags: "-run=TestWindowPostDispute"
winpost-test: "1"
test-suite-name: window-post-dispute
- test-terminate:
codecov-upload: true
go-test-flags: "-run=TestTerminate"
winpost-test: "1"
test-suite-name: terminate
- test-deadline-toggling:
codecov-upload: true
go-test-flags: "-run=TestDeadlineToggling"
deadline-test: "1"
test-suite-name: deadline-toggling
- test-short:
go-test-flags: "--timeout 10m --short"
test-suite-name: short
filters:
tags:
only:
- /^v\d+\.\d+\.\d+(-rc\d+)?$/
- check-proofs-multicore-sdr:
codecov-upload: true
name: test-itest-api
suite: itest-api
target: "./itests/api_test.go"

- test:
name: test-itest-batch_deal
suite: itest-batch_deal
target: "./itests/batch_deal_test.go"

- test:
name: test-itest-ccupgrade
suite: itest-ccupgrade
target: "./itests/ccupgrade_test.go"

- test:
name: test-itest-cli
suite: itest-cli
target: "./itests/cli_test.go"

- test:
name: test-itest-deadlines
suite: itest-deadlines
target: "./itests/deadlines_test.go"

- test:
name: test-itest-deals
suite: itest-deals
target: "./itests/deals_test.go"

- test:
name: test-itest-gateway
suite: itest-gateway
target: "./itests/gateway_test.go"

- test:
name: test-itest-multisig
suite: itest-multisig
target: "./itests/multisig_test.go"

- test:
name: test-itest-paych_api
suite: itest-paych_api
target: "./itests/paych_api_test.go"

- test:
name: test-itest-paych_cli
suite: itest-paych_cli
target: "./itests/paych_cli_test.go"

- test:
name: test-itest-sdr_upgrade
suite: itest-sdr_upgrade
target: "./itests/sdr_upgrade_test.go"

- test:
name: test-itest-sector_pledge
suite: itest-sector_pledge
target: "./itests/sector_pledge_test.go"

- test:
name: test-itest-sector_terminate
suite: itest-sector_terminate
target: "./itests/sector_terminate_test.go"

- test:
name: test-itest-tape
suite: itest-tape
target: "./itests/tape_test.go"

- test:
name: test-itest-verifreg
suite: itest-verifreg
target: "./itests/verifreg_test.go"

- test:
name: test-itest-wdpost_dispute
suite: itest-wdpost_dispute
target: "./itests/wdpost_dispute_test.go"

- test:
name: test-itest-wdpost
suite: itest-wdpost
target: "./itests/wdpost_test.go"

- test:
name: test-unit-cli
suite: utest-unit-cli
target: "./cli/... ./cmd/... ./api/..."
- test:
name: test-unit-node
suite: utest-unit-node
target: "./node/..."
- test:
name: test-unit-rest
suite: utest-unit-rest
target: "./blockstore/... ./build/... ./journal/... ./cli/... ./conformance/... ./node/... ./paychmgr/... ./api/... ./chain/... ./lib/... ./cmd/... ./extern/... ./gateway/... ./markets/... ./storage/... ./tools/..."
- test:
name: test-unit-storage
suite: utest-unit-storage
target: "./storage/... ./extern/..."
- test:
go-test-flags: "-run=TestMulticoreSDR"
test-suite-name: multicore-sdr-check
packages: "./extern/sector-storage/ffiwrapper"
suite: multicore-sdr-check
target: "./extern/sector-storage/ffiwrapper"
proofs-log-test: "1"
- test-conformance:
test-suite-name: conformance
packages: "./conformance"
suite: conformance
codecov-upload: false
target: "./conformance"
- test-conformance:
name: test-conformance-bleeding-edge
test-suite-name: conformance-bleeding-edge
packages: "./conformance"
codecov-upload: false
suite: conformance-bleeding-edge
target: "./conformance"
vectors-branch: master
- trigger-testplans:
filters:
Expand All @@ -842,37 +869,27 @@ workflows:
- master
- build-debug
- build-all:
requires:
- test-short
filters:
tags:
only:
- /^v\d+\.\d+\.\d+(-rc\d+)?$/
- build-ntwk-calibration:
requires:
- test-short
filters:
tags:
only:
- /^v\d+\.\d+\.\d+(-rc\d+)?$/
- build-ntwk-butterfly:
requires:
- test-short
filters:
tags:
only:
- /^v\d+\.\d+\.\d+(-rc\d+)?$/
- build-ntwk-nerpa:
requires:
- test-short
filters:
tags:
only:
- /^v\d+\.\d+\.\d+(-rc\d+)?$/
- build-lotus-soup
- build-macos:
requires:
- test-short
filters:
branches:
ignore:
Expand All @@ -881,8 +898,6 @@ workflows:
only:
- /^v\d+\.\d+\.\d+(-rc\d+)?$/
- build-appimage:
requires:
- test-short
filters:
branches:
ignore:
Expand Down
Loading