Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: Null buffer accounts for offset in substring kernel. #1704

Merged
merged 4 commits into from
May 16, 2022

Conversation

HaoYang670
Copy link
Contributor

Which issue does this PR close?

Closes #1639.

Rationale for this change

The null buffer was not aligned with the offset of array.

What changes are included in this PR?

Use bit_slice to align the null buffer.
Add tests to test the alignment.

Are there any user-facing changes?

No.

@github-actions github-actions bot added the arrow Changes to the arrow crate label May 16, 2022
@codecov-commenter
Copy link

Codecov Report

Merging #1704 (2840b5a) into master (ff182f1) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1704      +/-   ##
==========================================
+ Coverage   83.27%   83.29%   +0.01%     
==========================================
  Files         195      195              
  Lines       55937    55981      +44     
==========================================
+ Hits        46584    46629      +45     
+ Misses       9353     9352       -1     
Impacted Files Coverage Δ
arrow/src/compute/kernels/substring.rs 99.41% <100.00%> (+0.05%) ⬆️
arrow/src/array/transform/mod.rs 86.80% <0.00%> (-0.12%) ⬇️
parquet_derive/src/parquet_field.rs 66.21% <0.00%> (+0.22%) ⬆️
arrow/src/datatypes/datatype.rs 65.45% <0.00%> (+0.36%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ff182f1...2840b5a. Read the comment docs.

Copy link
Member

@viirya viirya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @HaoYang670

@alamb alamb changed the title Fix: Null buffer should be aligned with offset in substring kernel. Fix: Null buffer accounts for offset in substring kernel. May 16, 2022
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @HaoYang670

@alamb
Copy link
Contributor

alamb commented May 16, 2022

(nice tests)

@alamb alamb merged commit 8825d38 into apache:master May 16, 2022
@HaoYang670 HaoYang670 deleted the offset_substring_bug branch May 16, 2022 12:20
@HaoYang670
Copy link
Contributor Author

Thank you for your review @viirya @alamb !!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bug (substring kernel): The null buffer is not aligned when offset != 0
4 participants