Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed error in rle decoding #207

Merged
merged 1 commit into from
Dec 12, 2022

Conversation

ritchie46
Copy link
Collaborator

fixes jorgecarleitao/arrow2#1327

@jorgecarleitao could we patch this one? I think it is serious enough.

I'd like to add a test for this one, but I don't (yet) know parquet enough to to setup these conditions

@@ -54,7 +54,7 @@ fn read_next<'a, 'b>(decoder: &'b mut Decoder<'a>, remaining: usize) -> Result<S
});
let value = u32::from_le_bytes(bytes);
if additional == 1 {
State::Single(value)
State::Single(Some(value))
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because we have this small number optimization, the stop iteration call indicated by returning None was never triggered.

By keeping a Option<u32> state we keep track of this and return None once a value is already taken.

@jorgecarleitao jorgecarleitao changed the title fix rle decoding Fixed error in rle decoding Dec 12, 2022
@jorgecarleitao jorgecarleitao merged commit c7ae7f7 into jorgecarleitao:main Dec 12, 2022
@jorgecarleitao jorgecarleitao added the bug Something isn't working label Dec 12, 2022
jorgecarleitao pushed a commit that referenced this pull request Dec 12, 2022
@jorgecarleitao
Copy link
Owner

jorgecarleitao commented Dec 12, 2022

Thanks! Released in v0.17.1 through backporting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Parquet file with utf8 data is read incorrectly.
2 participants