Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement string representation for PartitionFilter #2183

Merged
merged 2 commits into from
Feb 14, 2024

Conversation

sonhmai
Copy link
Contributor

@sonhmai sonhmai commented Feb 12, 2024

Description

Implement string representation for PartitionFilter and add it to optimize command commit info for a TODO.

Following json representations of CommitInfo from delta scala project were found:

"operationParameters":{"predicate":"[\"(id#480 < 50)\"]"}
"operationParameters":{"predicate":"[\"(col1#1939 = 2)\"]"}
"operationParameters":{"predicate":"[\"`col2` LIKE 'data-2-%'\"]"}
"operationParameters":{"predicate":"[\"(id#378L < 2)\"]"}
"operationParameters":{
  "predicate":"[\"(spark_catalog.delta.`/table-uuid`.id IN (0, 180, 308, 225, 756, 1007, 1503))\"]"
}

So I think representing predicate as a list of filters in string representation would be logical, for example:

// single PartitionFilter
"operationParameters":{"predicate":"[\"date = '2022-05-22'\"]"}
// two filters
"operationParameters":{"predicate":"[\"date = '2022-05-22'\", \"country = 'US'\"]"}

Related Issue(s)

implement TODO in #607

@github-actions github-actions bot added the binding/rust Issues for the Rust crate label Feb 12, 2024
@rtyler rtyler requested a review from ion-elgreco February 13, 2024 16:21
@sonhmai
Copy link
Contributor Author

sonhmai commented Feb 14, 2024

@rtyler @ion-elgreco: oops, sorry there were test fix that was local. I pushed them.

@ion-elgreco ion-elgreco force-pushed the rust/impl-todo-optimize-test branch from 73b779d to 8fd84e4 Compare February 14, 2024 09:51
Copy link
Collaborator

@ion-elgreco ion-elgreco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nicee, thanks!! @sonhmai

@ion-elgreco ion-elgreco merged commit 47de67f into delta-io:main Feb 14, 2024
20 checks passed
@sonhmai sonhmai deleted the rust/impl-todo-optimize-test branch February 15, 2024 04:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
binding/rust Issues for the Rust crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants