Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve documentation for ExprVisitor, port simple uses to new walking function #4916

Merged
merged 2 commits into from
Jan 18, 2023

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Jan 15, 2023

Which issue does this PR close?

N/A

Rationale for this change

  1. I want to make the code easier to understand (and thus lower the barrier to using DataFusion for new comers -- see Update main DataFusion README #4903 )
  2. I enjoy making the code simpler to use

Follow on to #4906

Some uses of ExpressionVisitor need its full power (visit both on pre/post traversal, see https://github.com/apache/arrow-datafusion/blob/d49c805c93498b542c130c77099b719f2a4ea8ba/datafusion/optimizer/src/common_subexpr_eliminate.rs#L421) but several uses just need a simple town down traversal.

#4906 added a convenience function to do that traversal.

What changes are included in this PR?

  1. Cleans up doc comments,
  2. renames walk_expr_down to inspect_expr_pre
  3. rewrites some other uses of ExpressionVisitor to use inspect_expr_pre to simplify them

I am hoping to do something similar to ExpressionMutator for rewriting exprs.

Are these changes tested?

Covered by existing tests

@github-actions github-actions bot added logical-expr Logical plan and expressions optimizer Optimizer rules labels Jan 15, 2023
@@ -83,20 +85,16 @@ pub fn grouping_set_to_exprlist(group_expr: &[Expr]) -> Result<Vec<Expr>> {
}
}

/// Recursively walk an expression tree, collecting the unique set of column names
/// Recursively walk an expression tree, collecting the unique set of columns
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here is a pretty good example of reducing the ceremony required do to a walk of an expr by using inspect_expr_pre rather than having to define a visitor explicitly

@jackwener jackwener self-requested a review January 17, 2023 05:24
Copy link
Member

@jackwener jackwener left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look great to me.
Sorry for review it too late. I'm really busy to working recently....

Comment on lines +90 to +91
pub fn expr_to_columns(expr: &Expr, accum: &mut HashSet<Column>) -> Result<()> {
inspect_expr_pre(expr, |expr| {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

love this change❤️

@alamb
Copy link
Contributor Author

alamb commented Jan 18, 2023

Look great to me.
Sorry for review it too late. I'm really busy to working recently....

Thanks @jackwener -- I totally understand!

@alamb alamb merged commit ef4ca7e into apache:master Jan 18, 2023
@ursabot
Copy link

ursabot commented Jan 18, 2023

Benchmark runs are scheduled for baseline = ba9fc12 and contender = ef4ca7e. ef4ca7e is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ec2-t3-xlarge-us-east-2] ec2-t3-xlarge-us-east-2
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on test-mac-arm] test-mac-arm
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ursa-i9-9960x] ursa-i9-9960x
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ursa-thinkcentre-m75q] ursa-thinkcentre-m75q
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

@alamb alamb deleted the alamb/easier_expr_traversal branch July 26, 2024 10:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
logical-expr Logical plan and expressions optimizer Optimizer rules
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants