Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(planner): improve push down filter join #14872

Merged
merged 61 commits into from
Mar 27, 2024

Conversation

Dousir9
Copy link
Member

@Dousir9 Dousir9 commented Mar 7, 2024

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

  1. Introduce can_filter_null to convert more outer joins into inner joins
  2. Improve InferFilterOptimizer to support predicate derivation of outer/semi/anti join, for example, for sql select * from t1 left join t2 on t1.a = t2.a where t1.a > 0, we can get new filter t2.a > 0 and use it to filter out more hash join build side data in advance;
  • Fixes #[Link the issue here]

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@Dousir9 Dousir9 changed the title feat: improve push down filter join feat(planner): improve push down filter join Mar 7, 2024
@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Mar 7, 2024
@Dousir9 Dousir9 requested review from leiysky and xudong963 and removed request for leiysky March 18, 2024 17:19
@Dousir9 Dousir9 marked this pull request as ready for review March 18, 2024 17:19
@Dousir9 Dousir9 added this pull request to the merge queue Mar 27, 2024
Merged via the queue into databendlabs:main with commit 350d24f Mar 27, 2024
72 checks passed
@Dousir9 Dousir9 deleted the refactor_push_down_filter_join branch March 27, 2024 05:44
yufan022 added a commit to yufan022/databend that referenced this pull request Apr 25, 2024
yufan022 pushed a commit to yufan022/databend that referenced this pull request Apr 25, 2024
* feat: improve push down filter join

* fix: reverse operator when (constant, column_ref)

* chore: add more test

* chore: complete sqllogictest

* fix: outer join push down

* chore: remove z3 and add new method

* chore: move

* chore: refactor try push down filter join

* chore: update sqllogictest

* chore: update sqllogictest

* chore: update sqllogictest

* chore: fix push down single join

* chore: update subquery

* fix: after join reorder don not push down

* chore: update native

* chore: improve can_filter_null

* chore: fix test

* feat: improve infer and / or filter

* chore: update native sqllogictest

* chore: update sqllogictest

* feat: add SingleToInnerOptimizer

* chore: remove after_join_reorder

* chore: fix outer join to inner join

* test: update parquet explain sqllogictest

* test: update native explain sqllogictest

* chore: make lint code

* chore: fix Cascades optimizer

* chore: add z3-prove

* chore: update sqllogictest

* chore: fix infer filter for outer join

* chore: update sqllogictest

* chore: update ee test

* chore: fix tpcds test

* chore: update sqllogictest

* chore: update ee test

* chore: fix distributed query plan

* chore: update sqllogictet

* chore: update test comment

* chore: add comments

* chore: make lint

* chore: update sqllogictest

* chore: suuport more join type

* chore: refine code

* chore: add semi / anti test

* chore: update sqllogictest

* chore: improve infer filter

* chore: update sqllogictest

* chore: add more comments and remove useless code

* chore: make lint

* test: update memo sqllogictest

* chore: update memo test

* chore: update ee test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-feature this PR introduces a new feature to the codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants