Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to Wasmer Reborn (part 1) #504

Merged
merged 128 commits into from
Dec 10, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
128 commits
Select commit Hold shift + click to select a range
d0bfe86
Update Wasmer dependencies
webmaster128 Sep 10, 2020
5aef528
Disable middleware
webmaster128 Aug 5, 2020
de354aa
Adapt code to new API
webmaster128 Aug 5, 2020
94102a7
Update to latest master of Wasmer
webmaster128 Sep 8, 2020
cadafc5
Update Wasmer error handling
webmaster128 Sep 8, 2020
cc94242
Make VM compile (no iterator or tests)
webmaster128 Sep 14, 2020
020ad3b
Replace try_into! with ?
webmaster128 Sep 15, 2020
b4805cc
Update Lockfile to latest Wasmer master
webmaster128 Sep 15, 2020
c92ed36
Merge branch 'master' into upgrade-to-wasmer-reborn
webmaster128 Oct 6, 2020
88f2053
Merge branch 'master' into upgrade-to-wasmer-reborn
webmaster128 Oct 6, 2020
cd9d05b
Add FfiError::IteratorDoesNotExist
webmaster128 Oct 7, 2020
ea96f77
Replace range with scan/next in Storage trait
webmaster128 Oct 6, 2020
786719f
Update MockStorage to new trait
webmaster128 Oct 6, 2020
fad1b70
Use with_storage_from_context in do_next
webmaster128 Oct 6, 2020
8605e14
Remove with_iterator_from_context
webmaster128 Oct 6, 2020
ae4cc8b
Remove VmError::IteratorDoesNotExist
webmaster128 Oct 6, 2020
069a6a5
Remove iterators from ContextData
webmaster128 Oct 6, 2020
0bf8d4d
Implement AddAssign for GasInfo
webmaster128 Oct 6, 2020
1c648c7
Handle errors in self.next perperly
webmaster128 Oct 7, 2020
b14e190
Adapt burner integration tests to new storage
webmaster128 Oct 7, 2020
c90020b
Add note on iterator IDs
webmaster128 Oct 7, 2020
a40eb0e
Remove StorageIterator and MockIterator
webmaster128 Oct 7, 2020
5fe4cbe
Add CHANGELOG entries
webmaster128 Oct 12, 2020
30a3e82
Remove outdated doc comment
webmaster128 Oct 12, 2020
a709584
Bump wasmer version in lockfile
webmaster128 Oct 12, 2020
2996a65
Merge branch 'replace-range' into upgrade-to-wasmer-reborn-new-storage
webmaster128 Oct 12, 2020
4efb55f
Update a lot of tests
webmaster128 Oct 12, 2020
370bc5c
Let with_{func,querier,storage}_from_context take env references
webmaster128 Oct 12, 2020
5b57ab3
Avoid MockApi::new
webmaster128 Oct 12, 2020
9856ae4
Resolve more test problems
webmaster128 Oct 12, 2020
233655e
Merge branch 'master' into upgrade-to-wasmer-reborn-new-storage
webmaster128 Oct 12, 2020
e8fdb4c
Remove unused imports
webmaster128 Oct 12, 2020
ed031f3
Use memory from instance in Env
webmaster128 Oct 13, 2020
452e0c0
Remove unnecessary &
webmaster128 Oct 13, 2020
7863c5f
Adapt backend tests
webmaster128 Oct 13, 2020
91b0f4c
Remove unused variables
webmaster128 Oct 13, 2020
0ed43d7
Remove pointless clones
webmaster128 Oct 13, 2020
9761fc3
Re-enable test set_storage_readonly_works
webmaster128 Oct 13, 2020
6a3554a
Move pointers to block where they are used
webmaster128 Oct 13, 2020
6a282b5
Manually create import object
webmaster128 Oct 13, 2020
737bfcf
Add constructor function for Env
webmaster128 Oct 13, 2020
9bc3a57
Move with_func_from_context/with_storage_from_context/with_querier_fr…
webmaster128 Oct 13, 2020
8816b61
move set_wasmer_instance into env
webmaster128 Oct 13, 2020
a85533f
Move is_storage_readonly/set_storage_readonly to env
webmaster128 Oct 13, 2020
f401ab8
Remove obsolete where clauses
webmaster128 Oct 13, 2020
e73bf2d
Remove unused imports
webmaster128 Oct 14, 2020
65b394b
Use new_native_with_env instead of closures where possible
webmaster128 Oct 14, 2020
d5bf5a6
Move WASM_PAGE_SIZE to tests
webmaster128 Oct 14, 2020
96767f2
Merge branch 'master' into upgrade-to-wasmer-reborn-new-storage
webmaster128 Oct 14, 2020
02c6029
Fix iterator tests
webmaster128 Oct 27, 2020
899968c
Merge branch 'master' into upgrade-to-wasmer-reborn-new-storage
webmaster128 Oct 27, 2020
8bd2d50
Use Wasmer from Github
webmaster128 Oct 27, 2020
f5e7d75
Remove cranelift backend
webmaster128 Oct 28, 2020
13ddda4
Organize wasm_backend code
webmaster128 Oct 28, 2020
cbfa0b5
Remove backend
webmaster128 Oct 28, 2020
0fdd63a
Store store in FileSystemCache
webmaster128 Oct 28, 2020
709699c
Update wasmer
webmaster128 Oct 28, 2020
24b34f3
Add LimitingTunables
webmaster128 Oct 28, 2020
57ee49a
Create make_store/make_store_headless and introduce a memory limit
webmaster128 Oct 28, 2020
c413bfe
Update wasmer
webmaster128 Oct 28, 2020
c52a60a
Split memory adjustment and memory validation
webmaster128 Oct 28, 2020
3f63c99
Reuse store from Module
webmaster128 Oct 29, 2020
563ac1a
Merge branch 'master' into upgrade-to-wasmer-reborn
webmaster128 Oct 29, 2020
b1864f4
Add memory_limit to InstanceOptions
webmaster128 Oct 29, 2020
8821fe9
Fix linting issues
webmaster128 Oct 29, 2020
7c07d83
Merge branch 'master' into upgrade-to-wasmer-reborn
webmaster128 Oct 30, 2020
88ce885
Add testing script
webmaster128 Nov 11, 2020
804b2ae
upgrade Wasmer to 1.0.0-alpha5
webmaster128 Nov 11, 2020
a6ab951
Merge branch 'master' into upgrade-to-wasmer-reborn
webmaster128 Nov 11, 2020
52c65c2
Merge branch 'master' into upgrade-to-wasmer-reborn
webmaster128 Nov 24, 2020
7dcf3c3
Update Wasmer to 92ee6974
webmaster128 Nov 24, 2020
1cea103
Make Env immutable
webmaster128 Nov 24, 2020
dcac4ae
Upgrade Wasmer to 92ee6974
webmaster128 Nov 24, 2020
baf5570
Merge branch 'master' into upgrade-to-wasmer-reborn
webmaster128 Nov 24, 2020
34fc958
Test contracts as part of reborn_testing
webmaster128 Nov 26, 2020
318f717
Update lockfiles to latest Wasmer crates
webmaster128 Nov 26, 2020
ec4c9bb
Update Wasmer to 6041c031b9
webmaster128 Dec 1, 2020
c541dd5
Test results of call_function
webmaster128 Dec 1, 2020
106550a
Add calls tests
webmaster128 Dec 1, 2020
bab0b46
Merge branch 'master' into upgrade-to-wasmer-reborn
webmaster128 Dec 1, 2020
b9b3786
Organize instance and cache testing
webmaster128 Dec 1, 2020
837f90d
Convert memory limit to Size and add to MockInstanceOptions
webmaster128 Dec 1, 2020
343c49d
Fix and activate test allocate_deallocate_works
webmaster128 Dec 1, 2020
b8dc142
Ensure instance lives long enough
webmaster128 Dec 1, 2020
6e19660
Fix context:: testing
webmaster128 Dec 1, 2020
5b0f1ba
Make all cache:: and instance:: tests pass
webmaster128 Dec 1, 2020
ec53268
Exclude gas tests without metering
webmaster128 Dec 1, 2020
65f804a
Fix imports tests
webmaster128 Dec 1, 2020
ab2383e
Run clippy via Rust stable
webmaster128 Dec 1, 2020
2880169
Remove package_vm_cranelift
webmaster128 Dec 1, 2020
ab26716
Use stable Rust in package_vm
webmaster128 Dec 1, 2020
47e4646
Remove job contract_hackatom_in_singlepass_vm
webmaster128 Dec 1, 2020
c999bcb
Update Wasmer to 1.0.0-beta1
webmaster128 Dec 2, 2020
9b80705
Remove dependency memoffset 0.5
webmaster128 Dec 2, 2020
d76d490
Remove explicit wasmer-types dependency
webmaster128 Dec 4, 2020
971fcc1
Use error types from wasmer
webmaster128 Dec 4, 2020
35baeaa
Reduce features and dependencies of wasmer
webmaster128 Dec 4, 2020
17a54c4
Use Singlepass and JIT via wasmer crate
webmaster128 Dec 4, 2020
acdcf67
Use Engine from wasmer crate and document Tunables problem
webmaster128 Dec 4, 2020
4f0b8a1
Merge branch 'master' into upgrade-to-wasmer-reborn
webmaster128 Dec 4, 2020
e8efecb
Update wasmer and remove explicit wasmer-compiler dependency
webmaster128 Dec 4, 2020
0319815
Merge branch 'master' into upgrade-to-wasmer-reborn
webmaster128 Dec 4, 2020
7100c86
Update Wasmer to e9529c2c86
webmaster128 Dec 6, 2020
def0ff7
Use display representation of Wasmer errors
webmaster128 Dec 6, 2020
a77ee2d
Use Cranelift for integration testing
webmaster128 Dec 7, 2020
231af85
Merge branch 'master' into upgrade-to-wasmer-reborn
webmaster128 Dec 7, 2020
81d281b
Update Wasmer to 9fc32dfd0f3
webmaster128 Dec 8, 2020
48730f3
Run all tests in local testing
webmaster128 Dec 8, 2020
63e7207
Remove usages of Rust nightly
webmaster128 Dec 8, 2020
1d9dae1
Remove temporary testing script
webmaster128 Dec 8, 2020
10fdde0
Fix cosmwasm-std version after merge conflict
webmaster128 Dec 8, 2020
6004449
Update VM's Testing docs
webmaster128 Dec 8, 2020
fde0d32
Avoid double references to Memory
webmaster128 Dec 8, 2020
2cba528
Merge branch 'master' into upgrade-to-wasmer-reborn
webmaster128 Dec 9, 2020
0692deb
Update Wasmer to e1c92737
webmaster128 Dec 9, 2020
93fe981
Merge branch 'master' into upgrade-to-wasmer-reborn
webmaster128 Dec 9, 2020
3c23c6d
Merge branch 'master' into upgrade-to-wasmer-reborn
webmaster128 Dec 9, 2020
707cc88
Fix job name package_vm_singlepass -> package_vm
webmaster128 Dec 9, 2020
3a602a7
Set RUST_BACKTRACE in benchmarking job and fix gas issue in CI
webmaster128 Dec 9, 2020
c74d978
Add ref_to_u32
webmaster128 Dec 9, 2020
54dacc9
Use ref_to_u32 for Wasmer's Value
webmaster128 Dec 9, 2020
6551573
Make memory limit optional to avoid the need for a fake limit
webmaster128 Dec 10, 2020
2f8f86d
Use a single immutable env reference
webmaster128 Dec 10, 2020
81381c1
Streamline callback and return generic types
webmaster128 Dec 10, 2020
8f0815d
Rename module context -> environment
webmaster128 Dec 10, 2020
a5194fd
Rename to move_into_environment/move_out_of_environment
webmaster128 Dec 10, 2020
3f0e17e
Rename Env -> Environment to avoid conflict with Env in contracts
webmaster128 Dec 10, 2020
e657e59
Add print_debug to Environment
webmaster128 Dec 10, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
147 changes: 52 additions & 95 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,26 +7,26 @@ workflows:
- package_schema
- package_std
- package_storage
- package_vm
- package_vm_cranelift
- package_vm_singlepass
- contract_burner
- contract_hackatom
- contract_hackatom_in_singlepass_vm
- contract_queue
- contract_reflect
- contract_staking
- fmt
- clippy
- benchmarking:
requires:
- package_vm
- package_vm_cranelift
- package_vm_singlepass
filters:
branches:
only:
- master
# 👇 Add your branch here if benchmarking matters to your work
- benchmarking
- upgrade-to-wasmer-reborn
deploy:
jobs:
- build_and_upload_devcontracts:
Expand Down Expand Up @@ -154,7 +154,7 @@ jobs:
- target/debug/deps
key: cargocache-v2-package_storage-rust:1.47.0-{{ checksum "Cargo.lock" }}

package_vm_cranelift:
package_vm:
docker:
- image: rust:1.47.0
steps:
Expand All @@ -164,57 +164,65 @@ jobs:
command: rustc --version; cargo --version; rustup --version; rustup target list --installed
- restore_cache:
keys:
- cargocache-v2-package_vm_cranelift-rust:1.47.0-{{ checksum "Cargo.lock" }}
- cargocache-v2-package_vm-rust:1.47.0-{{ checksum "Cargo.lock" }}
- run:
name: Build
working_directory: ~/project/packages/vm
command: cargo build --locked --no-default-features --features default-cranelift,iterator
command: cargo build --locked
- run:
name: Build with iterator
working_directory: ~/project/packages/vm
command: cargo build --locked --features iterator
- run:
name: Test
working_directory: ~/project/packages/vm
command: cargo test --locked --no-default-features --features default-cranelift,iterator
command: cargo test --locked
- run:
name: Test with iterator
working_directory: ~/project/packages/vm
command: cargo test --locked --features iterator
- save_cache:
paths:
- /usr/local/cargo/registry
- target/debug/.fingerprint
- target/debug/build
- target/debug/deps
key: cargocache-v2-package_vm_cranelift-rust:1.47.0-{{ checksum "Cargo.lock" }}
key: cargocache-v2-package_vm-rust:1.47.0-{{ checksum "Cargo.lock" }}

package_vm_singlepass:
package_vm_cranelift:
docker:
- image: rustlang/rust:nightly
- image: rust:1.47.0
steps:
- checkout
- run:
name: Version information
command: rustc --version; cargo --version; rustup --version; rustup target list --installed
- restore_cache:
keys:
- cargocache-v2-package_vm_singlepass-rust:nightly-{{ checksum "Cargo.lock" }}
- cargocache-v2-package_vm_cranelift-rust:1.47.0-{{ checksum "Cargo.lock" }}
- run:
name: Build
working_directory: ~/project/packages/vm
command: cargo build --locked
command: cargo build --locked --features cranelift
- run:
name: Build with iterator
working_directory: ~/project/packages/vm
command: cargo build --locked --features iterator
command: cargo build --locked --features cranelift,iterator
- run:
name: Test
working_directory: ~/project/packages/vm
command: cargo test --locked
command: cargo test --locked --features cranelift
- run:
name: Test with iterator
working_directory: ~/project/packages/vm
command: cargo test --locked --features iterator
command: cargo test --locked --features cranelift,iterator
- save_cache:
paths:
- /usr/local/cargo/registry
- target/debug/.fingerprint
- target/debug/build
- target/debug/deps
key: cargocache-v2-package_vm_singlepass-rust:nightly-{{ checksum "Cargo.lock" }}
key: cargocache-v2-package_vm_cranelift-rust:1.47.0-{{ checksum "Cargo.lock" }}

contract_burner:
docker:
Expand All @@ -241,8 +249,11 @@ jobs:
name: Unit tests
command: cargo unit-test --locked
- run:
name: Integration tests
name: Integration tests (cranelift backend)
command: cargo integration-test --locked
- run:
name: Integration tests (singlepass backend)
command: cargo integration-test --locked --no-default-features
- run:
name: Build and run schema generator
command: cargo schema --locked
Expand Down Expand Up @@ -291,8 +302,11 @@ jobs:
name: Unit tests
command: cargo unit-test --locked
- run:
name: Integration tests
name: Integration tests (cranelift backend)
command: cargo integration-test --locked
- run:
name: Integration tests (singlepass backend)
command: cargo integration-test --locked --no-default-features
- run:
name: Build and run schema generator
command: cargo schema --locked
Expand All @@ -316,43 +330,6 @@ jobs:
- target/wasm32-unknown-unknown/release/deps
key: cargocache-v2-contract_hackatom-rust:1.47.0-{{ checksum "Cargo.lock" }}

# In this job we use singlepass as the VM to execute integration tests. This requires Rust nightly.
# Avoid duplicating generic checks like unit tests or schema generation – they belong in the generic hackatom job.
contract_hackatom_in_singlepass_vm:
docker:
- image: rustlang/rust:nightly
environment:
RUST_BACKTRACE: 1
working_directory: ~/cosmwasm/contracts/hackatom
steps:
- checkout:
path: ~/cosmwasm
- run:
name: Version information
command: rustc --version; cargo --version; rustup --version
- restore_cache:
keys:
- cargocache-v2-contract_hackatom_in_singlepass_vm-rust:nightly-{{ checksum "Cargo.lock" }}
- run:
name: Add wasm32 target
command: rustup target add wasm32-unknown-unknown && rustup target list --installed
- run:
name: Build wasm binary
command: cargo wasm --locked
- run:
name: Integration tests
command: cargo integration-test --no-default-features --features singlepass --locked
- save_cache:
paths:
- /usr/local/cargo/registry
- target/debug/.fingerprint
- target/debug/build
- target/debug/deps
- target/wasm32-unknown-unknown/release/.fingerprint
- target/wasm32-unknown-unknown/release/build
- target/wasm32-unknown-unknown/release/deps
key: cargocache-v2-contract_hackatom_in_singlepass_vm-rust:nightly-{{ checksum "Cargo.lock" }}

contract_queue:
docker:
- image: rust:1.47.0
Expand All @@ -378,8 +355,11 @@ jobs:
name: Unit tests
command: cargo unit-test --locked
- run:
name: Integration tests
name: Integration tests (cranelift backend)
command: cargo integration-test --locked
- run:
name: Integration tests (singlepass backend)
command: cargo integration-test --locked --no-default-features
- run:
name: Build and run schema generator
command: cargo schema --locked
Expand Down Expand Up @@ -428,8 +408,11 @@ jobs:
name: Unit tests
command: cargo unit-test --locked
- run:
name: Integration tests
name: Integration tests (cranelift backend)
command: cargo integration-test --locked
- run:
name: Integration tests (singlepass backend)
command: cargo integration-test --locked --no-default-features
- run:
name: Build and run schema generator
command: cargo schema --locked
Expand Down Expand Up @@ -478,8 +461,11 @@ jobs:
name: Unit tests
command: cargo unit-test --locked
- run:
name: Integration tests
name: Integration tests (cranelift backend)
command: cargo integration-test --locked
- run:
name: Integration tests (singlepass backend)
command: cargo integration-test --locked --no-default-features
- run:
name: Build and run schema generator
command: cargo schema --locked
Expand Down Expand Up @@ -551,23 +537,11 @@ jobs:
clippy:
docker:
- image: rust:1.47.0
environment:
# Make sure to choose version with clippy present: https://rust-lang.github.io/rustup-components-history/x86_64-unknown-linux-gnu.html
NIGHTLY_TOOLCHAIN: nightly-2020-09-08
steps:
- checkout
- run:
name: Install Rust nightly
command: |
rustup --version
rustup toolchain install $NIGHTLY_TOOLCHAIN --allow-downgrade --profile minimal --component clippy
rustup target list --installed
- run:
name: Version information (default; stable)
name: Version information
command: rustc --version && cargo --version
- run:
name: Version information (nightly)
command: rustc +$NIGHTLY_TOOLCHAIN --version && cargo +$NIGHTLY_TOOLCHAIN --version
- restore_cache:
keys:
- cargocache-v2-clippy-rust:1.47.0-{{ checksum "Cargo.lock" }}-{{ checksum "contracts/burner/Cargo.lock" }}-{{ checksum "contracts/hackatom/Cargo.lock" }}-{{ checksum "contracts/queue/Cargo.lock" }}-{{ checksum "contracts/reflect/Cargo.lock" }}-{{ checksum "contracts/staking/Cargo.lock" }}
Expand Down Expand Up @@ -598,21 +572,13 @@ jobs:
working_directory: ~/project/packages/storage
command: cargo clippy --features iterator -- -D warnings
- run:
name: Clippy linting on vm (singlepass; no feature flags)
name: Clippy linting on vm (no feature flags)
working_directory: ~/project/packages/vm
command: cargo +$NIGHTLY_TOOLCHAIN clippy -- -D warnings
- run:
name: Clippy linting on vm (singlepass; all feature flags)
working_directory: ~/project/packages/vm
command: cargo +$NIGHTLY_TOOLCHAIN clippy --features iterator -- -D warnings
- run:
name: Clippy linting on vm (cranelift; no feature flags)
working_directory: ~/project/packages/vm
command: cargo +$NIGHTLY_TOOLCHAIN clippy --no-default-features --features default-cranelift -- -D warnings
command: cargo clippy -- -D warnings
- run:
name: Clippy linting on vm (cranelift; all feature flags)
name: Clippy linting on vm (all feature flags)
working_directory: ~/project/packages/vm
command: cargo +$NIGHTLY_TOOLCHAIN clippy --no-default-features --features default-cranelift,iterator -- -D warnings
command: cargo clippy --features cranelift,iterator -- -D warnings
#
# Contracts
#
Expand Down Expand Up @@ -663,32 +629,23 @@ jobs:
docker:
- image: rust:1.47.0
environment:
NIGHTLY_TOOLCHAIN: nightly-2020-09-08
RUST_BACKTRACE: 1
steps:
- checkout
- run:
name: Install Rust nightly
command: |
rustup --version
rustup toolchain install $NIGHTLY_TOOLCHAIN --allow-downgrade --profile minimal --component clippy
rustup target list --installed
- run:
name: Version information (default; stable)
command: rustc --version && cargo --version
- run:
name: Version information (nightly)
command: rustc +$NIGHTLY_TOOLCHAIN --version && cargo +$NIGHTLY_TOOLCHAIN --version
- restore_cache:
keys:
- cargocache-v2-benchmarking-rust:1.47.0-{{ checksum "Cargo.lock" }}
- run:
name: Run benchmarks (Singlepass)
working_directory: ~/project/packages/vm
command: cargo +$NIGHTLY_TOOLCHAIN bench --no-default-features --features singlepass,default-singlepass -- --save-baseline singlepass
command: cargo bench --no-default-features -- --save-baseline singlepass
- run:
name: Run benchmarks (Cranelift)
working_directory: ~/project/packages/vm
command: cargo +$NIGHTLY_TOOLCHAIN bench --no-default-features --features cranelift,default-cranelift -- --save-baseline cranelift
command: cargo bench --no-default-features --features cranelift -- --save-baseline cranelift
- save_cache:
paths:
- /usr/local/cargo/registry
Expand Down
3 changes: 1 addition & 2 deletions .mergify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,10 @@ pull_request_rules:
- "status-success=ci/circleci: package_schema"
- "status-success=ci/circleci: package_std"
- "status-success=ci/circleci: package_storage"
- "status-success=ci/circleci: package_vm"
- "status-success=ci/circleci: package_vm_cranelift"
- "status-success=ci/circleci: package_vm_singlepass"
- "status-success=ci/circleci: contract_burner"
- "status-success=ci/circleci: contract_hackatom"
- "status-success=ci/circleci: contract_hackatom_in_singlepass_vm"
- "status-success=ci/circleci: contract_queue"
- "status-success=ci/circleci: contract_reflect"
- "status-success=ci/circleci: contract_staking"
Expand Down
Loading