You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
A clear and concise description of what the bug is.
Function read_record_batch deals with projection in arrow/src/ipc/reader.rs.
Current logic may not advance the node_index and buffer_index correctly because it does not call create_array for skipped nodes.
To Reproduce
Steps to reproduce the behavior:
[dependencies]
arrow = "14.0.0"
use std::fs::File;
use std::sync::Arc;
use arrow::array::*;
use arrow::datatypes::*;
use arrow::ipc::reader::*;
use arrow::ipc::writer::*;
use arrow::record_batch::RecordBatch;
fn main() {
let schema = Schema::new(vec![
Field::new("f0", DataType::UInt32, false),
Field::new("f1", DataType::Utf8, false),
Field::new("f2", DataType::Boolean, false),
]);
let array0 = UInt32Array::from(vec![1, 2, 3]);
let array1 = StringArray::from(vec!["foo", "bar", "baz"]);
let array2 = BooleanArray::from(vec![true, false, true]);
let record_batch = RecordBatch::try_new(
Arc::new(schema.clone()),
vec![Arc::new(array0), Arc::new(array1), Arc::new(array2)]
).unwrap();
{
let file = File::create("./test.arrow_file").unwrap();
let mut writer = FileWriter::try_new(file, &schema).unwrap();
writer.write(&record_batch).unwrap();
writer.finish().unwrap();
}
let projection = vec![1];
let file = File::open("./test.arrow_file").unwrap();
let arrow_reader = FileReader::try_new(file, Some(projection.clone()));
let batch = arrow_reader.unwrap().next();
println!("{:?}", batch);
}
Above code will produce:
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: InvalidArgumentError("First offset 1 in Utf8 is smaller than last offset 0")', /home/yupeng/.cargo/registry/src/github.com-1ecc6299db9ec823/arrow-14.0.0/src/ipc/reader.rs:287:29
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Describe the bug
A clear and concise description of what the bug is.
Function
read_record_batch
deals withprojection
inarrow/src/ipc/reader.rs
.Current logic may not advance the
node_index
andbuffer_index
correctly because it does not callcreate_array
for skipped nodes.To Reproduce
Steps to reproduce the behavior:
Above code will produce:
Expected behavior
Output:
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: