Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug (substring kernel): The null buffer is not aligned when offset != 0 #1639

Closed
Tracked by #1531
HaoYang670 opened this issue May 3, 2022 · 1 comment · Fixed by #1704
Closed
Tracked by #1531

Bug (substring kernel): The null buffer is not aligned when offset != 0 #1639

HaoYang670 opened this issue May 3, 2022 · 1 comment · Fixed by #1704
Labels
arrow Changes to the arrow crate bug

Comments

@HaoYang670
Copy link
Contributor

Describe the bug
Track #1633 (comment)
We have the bug in BinaryArray and StringArray in the substring kernel.

To Reproduce

    fn offset_fixed_size_binary() -> Result<()> {
        let values = *b"hellotherearrow";
        let offsets: [i32; 4] = [0, 5, 10, 15];
        // set the first and third element to be valid
        let bits_v = [0b101_u8];

        let data = ArrayData::builder(DataType::Binary)
            .len(2)
            .add_buffer(Buffer::from_slice_ref(&offsets))
            .add_buffer(Buffer::from_slice_ref(&values))
            .offset(1)
            .null_bit_buffer(Buffer::from(bits_v))
            .build()
            .unwrap();
        // array is `[null, "arrow"]`
        let array = BinaryArray::from(data);
        // result is `[null, "rrow"]`
        let result = substring(&array, 1, None)?;
        let result = result
            .as_any()
            .downcast_ref::<BinaryArray>()
            .unwrap();
        let expected = BinaryArray::from_opt_vec(
            vec![None, Some(b"rrow")],
        );
        assert_eq!(result, &expected);

        Ok(())
    }

Expected behavior

  1. fix the bug
  2. add tests

Additional context
Add any other context about the problem here.

@HaoYang670
Copy link
Contributor Author

blocked by #1665

@alamb alamb added parquet Changes to the parquet crate arrow Changes to the arrow crate and removed parquet Changes to the parquet crate labels May 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants