Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add sizeInBytes to _last_checkpoint and change size to # of actions #1477

Merged
merged 1 commit into from
Jun 20, 2023

Conversation

cmackenzie1
Copy link
Contributor

Description

The size field should be the number of actions stored in the checkpoint while sizeInBytes is used for the total size in bytes.

Added CheckPointBuilder to make the creation of these easier to use.

Related Issue(s)

Documentation

https://github.com/delta-io/delta/blob/master/PROTOCOL.md#last-checkpoint-file

@github-actions github-actions bot added binding/rust Issues for the Rust crate rust labels Jun 19, 2023
@github-actions
Copy link

ACTION NEEDED

delta-rs follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

@cmackenzie1 cmackenzie1 marked this pull request as ready for review June 20, 2023 15:33
wjones127
wjones127 previously approved these changes Jun 20, 2023
@wjones127 wjones127 changed the title fix: Add sizeInBytes to _last_checkpoint and change size to # of actions fix: add sizeInBytes to _last_checkpoint and change size to # of actions Jun 20, 2023
Copy link
Collaborator

@wjones127 wjones127 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the test is finding something is still wrong in the read path.

size: 0,
parts: None,
});
cp = Some(CheckPoint::new(version, 0, None));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wjones127 this was the culprit. Fixing...

Suggested change
cp = Some(CheckPoint::new(version, 0, None));
cp = Some(CheckPoint::new(curr_ver, 0, None));

…actions

The `size` field should be the number of actions stored in the checkpoint
while `sizeInBytes` is used for the total size in bytes.

Added `CheckPointBuilder` to make the creation of these easier to use.

- Closes delta-io#1468
@wjones127
Copy link
Collaborator

Thanks @cmackenzie1!

@wjones127 wjones127 merged commit e5dd8e2 into delta-io:main Jun 20, 2023
@cmackenzie1 cmackenzie1 deleted the cole/issue-1468 branch June 20, 2023 21:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
binding/rust Issues for the Rust crate rust
Projects
None yet
Development

Successfully merging this pull request may close these issues.

_last_checkpoint size field is incorrect
2 participants