Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add boolean, date, timestamp & binary partition types #1180

Merged
merged 7 commits into from
Mar 6, 2023

Conversation

marijncv
Copy link
Contributor

@marijncv marijncv commented Feb 26, 2023

Description

Adds boolean, date, timestamp & binary partition value types

Related Issue(s)

closes #1170

@github-actions github-actions bot added binding/rust Issues for the Rust crate rust labels Feb 26, 2023
Signed-off-by: Marijn Valk <[email protected]>
@marijncv marijncv marked this pull request as ready for review February 26, 2023 17:21
@marijncv marijncv changed the title add boolean, date & timestamp partition types add boolean, date, timestamp & binary partition types Feb 26, 2023
Signed-off-by: Marijn Valk <[email protected]>
Copy link
Collaborator

@wjones127 wjones127 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this @marijncv!

Just one comment, but otherwise looks great.

rust/src/writer/utils.rs Outdated Show resolved Hide resolved
@marijncv marijncv requested review from wjones127 and removed request for rtyler, xianwill, houqp, mosyp, fvaleye and roeap March 3, 2023 06:01
fvaleye
fvaleye previously approved these changes Mar 3, 2023
Copy link
Collaborator

@fvaleye fvaleye left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @marijncv
LGTM 👍

Copy link
Collaborator

@wjones127 wjones127 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay. I meant my earlier comment about not needing more than 1 value in the test to apply to all the test cases, not just the one I had made a suggestion on.

Added a few other comments to make the test cases a little more compact.

Comment on lines 300 to 311
(
Arc::new(Int8Array::from(vec![Some(1), Some(2)])),
Some(String::from("1")),
),
(
Arc::new(Int16Array::from(vec![Some(1), Some(2)])),
Some(String::from("1")),
),
(
Arc::new(Int32Array::from(vec![Some(1), Some(2)])),
Some(String::from("1")),
),
Copy link
Collaborator

@wjones127 wjones127 Mar 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be applied to all the test cases: You shouldn't need the Some inside the array.

Suggested change
(
Arc::new(Int8Array::from(vec![Some(1), Some(2)])),
Some(String::from("1")),
),
(
Arc::new(Int16Array::from(vec![Some(1), Some(2)])),
Some(String::from("1")),
),
(
Arc::new(Int32Array::from(vec![Some(1), Some(2)])),
Some(String::from("1")),
),
(Arc::new(Int8Array::from(vec![1])), Some("1")),
(Arc::new(Int16Array::from(vec![1])), Some("1")),
(Arc::new(Int32Array::from(vec![1])), Some("1")),


#[test]
fn test_stringified_partition_value() {
let reference_pairs: Vec<(Arc<dyn Array>, Option<String>)> = vec![
Copy link
Collaborator

@wjones127 wjones127 Mar 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use &str instead to avoid needing String::from() in every test case.

Suggested change
let reference_pairs: Vec<(Arc<dyn Array>, Option<String>)> = vec![
let reference_pairs: Vec<(Arc<dyn Array>, Option<&str>)> = vec![

),
];
for (vals, result) in reference_pairs {
assert_eq!(stringified_partition_value(&vals).unwrap(), result)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use as_deref() to turn Option<String> into Option<&str>:

Suggested change
assert_eq!(stringified_partition_value(&vals).unwrap(), result)
assert_eq!(
stringified_partition_value(&vals).unwrap().as_deref(),
result
)

Comment on lines 332 to 338
(
Arc::new(StringArray::from(vec![
Some(String::from("1")),
Some(String::from("2")),
])),
Some(String::from("1")),
),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The StringArray::from implementation can take a &str:

Suggested change
(
Arc::new(StringArray::from(vec![
Some(String::from("1")),
Some(String::from("2")),
])),
Some(String::from("1")),
),
(Arc::new(StringArray::from(vec!["1"])), Some("1")),

Copy link
Collaborator

@roeap roeap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Collaborator

@wjones127 wjones127 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Thanks @marijncv

@wjones127 wjones127 merged commit bb9f040 into delta-io:main Mar 6, 2023
chitralverma pushed a commit to chitralverma/delta-rs that referenced this pull request Mar 17, 2023
# Description
Adds boolean, date, timestamp & binary partition value types

# Related Issue(s)
closes delta-io#1170

---------

Signed-off-by: Marijn Valk <[email protected]>
Co-authored-by: Will Jones <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
binding/rust Issues for the Rust crate rust
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat(rust): support additional types for partition values
4 participants