Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

store partial encoded chunks #1975

Merged
merged 9 commits into from
Jan 21, 2020
Merged

store partial encoded chunks #1975

merged 9 commits into from
Jan 21, 2020

Conversation

Kouprin
Copy link
Member

@Kouprin Kouprin commented Jan 17, 2020

Fixes #1971.
Also fixes calculation of encoded_chunks.num_chunks_for_block().

@codecov
Copy link

codecov bot commented Jan 17, 2020

Codecov Report

Merging #1975 into staging will increase coverage by 0.38%.
The diff coverage is 85.54%.

Impacted file tree graph

@@             Coverage Diff             @@
##           staging    #1975      +/-   ##
===========================================
+ Coverage    87.17%   87.55%   +0.38%     
===========================================
  Files          167      167              
  Lines        31625    31759     +134     
===========================================
+ Hits         27569    27807     +238     
+ Misses        4056     3952     -104
Impacted Files Coverage Δ
chain/chunks/src/chunk_cache.rs 96.72% <100%> (+1.8%) ⬆️
chain/client/src/client_actor.rs 87.23% <100%> (ø) ⬆️
chain/client/src/client.rs 94.14% <100%> (+0.11%) ⬆️
chain/client/tests/catching_up.rs 75.92% <24.13%> (+75.92%) ⬆️
chain/chunks/src/lib.rs 93.47% <96.96%> (+0.18%) ⬆️
chain/client/tests/chunks_management.rs 96.94% <98.71%> (+0.91%) ⬆️
chain/chunks/src/types.rs 0% <0%> (-25%) ⬇️
chain/chain/src/store.rs 88.27% <0%> (-0.71%) ⬇️
chain/network/src/peer.rs 74.8% <0%> (-0.2%) ⬇️
...enesis-tools/genesis-csv-to-json/src/serde_with.rs 100% <0%> (ø) ⬆️
... and 36 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dd49b00...6ff6506. Read the comment docs.

chain/chunks/src/lib.rs Outdated Show resolved Hide resolved
chain/client/src/client.rs Outdated Show resolved Hide resolved
chain/client/tests/catching_up.rs Outdated Show resolved Hide resolved
@Kouprin Kouprin requested a review from SkidanovAlex January 17, 2020 19:29
chain/chunks/src/lib.rs Show resolved Hide resolved
chain/chunks/src/lib.rs Outdated Show resolved Hide resolved
@Kouprin Kouprin changed the base branch from chunks_grieving to staging January 18, 2020 11:53
@Kouprin Kouprin changed the base branch from staging to chunks_grieving January 18, 2020 11:54
chain/chunks/src/lib.rs Outdated Show resolved Hide resolved
chain/client/src/client.rs Outdated Show resolved Hide resolved
chain/client/tests/catching_up.rs Outdated Show resolved Hide resolved
.store_partial_encoded_chunk(&block.header, partial_encoded_chunk.clone());
assert_eq!(env.clients[0].shards_mgr.get_stored_partial_encoded_chunks(1).len(), 1);
assert_eq!(
env.clients[0].shards_mgr.get_stored_partial_encoded_chunks(1)[&0],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this work, you changed the hash but not the signature, right?
Or is it becaues signature verification is disabled in tests?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

verify_validator_signature is always true in KV-runtime.

@Kouprin Kouprin requested a review from SkidanovAlex January 18, 2020 15:52
chain/client/tests/catching_up.rs Outdated Show resolved Hide resolved
@@ -61,6 +63,9 @@ pub enum ProcessPartialEncodedChunkResult {
/// The Header is the header of the current chunk, which is unknown to the caller, to request
/// parts / receipts for
NeedMorePartsOrReceipts(ShardChunkHeader),
/// PartialEncodedChunkMessage is received earlier than Block for the same height.
/// Without the block we cannot restore the epoch and save encoded chunk data.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is epoch relevant here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In process_partial_encoded_chunk we run get_epoch_id_from_prev_block. This means we can't get epoch without having received Block and cannot store the chunk. In this case process_partial_encoded_chunk returns NeedBlock.

chain/client/src/client.rs Outdated Show resolved Hide resolved
@Kouprin Kouprin changed the base branch from chunks_grieving to staging January 21, 2020 11:24
@Kouprin Kouprin merged commit 51c7934 into staging Jan 21, 2020
@Kouprin Kouprin deleted the T1971 branch January 21, 2020 11:54
SkidanovAlex pushed a commit that referenced this pull request Jan 29, 2020
* store partial encoded chunks

* num_chunks_for_block fixing

* fixes

* fixes and sanity test

* minor fixes

* processing accepted_blocks after process_partial_encoded_chunk

* increasing max_wait_ms

* removed into_iter in accepted blocks

* verbose
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants