Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-50357][Core]If a stage contains ExpandExec, the CoalesceShuffl… #48819

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[SPARK-50357][Core]If a stage contains ExpandExec, the CoalesceShuffl… #48819

wants to merge 1 commit into from
+73 −1

Conversation

guixiaowen
Copy link
Contributor

…ePartitions rule will not be adjusted during the AQE phase

What changes were proposed in this pull request?

In the Aqe scenario, the optimization phase includes the ExpandEXEC scenario and does not perform partition merging.
For example

`SELECT

   /*+ SHUFFLE_MERGE(b) */

   s_date,

   sum(s_quantity * i_price) AS total_sales

FROM

   sales a

   JOIN items b ON s_item_id = i_item_id

WHERE
i_price < 10
GROUP BY
s_date with rollup;`

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Was this patch authored or co-authored using generative AI tooling?

[SPARK-50357][Core]If a stage contains ExpandExec, the CoalesceShuffl…
6e57bda 
…ePartitions rule will not be adjusted during the AQE phase
@github-actions github-actions bot added the SQL label Nov 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
SQL
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@guixiaowen