Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat(optimizer): propagate constants #2386

Merged
merged 16 commits into from
Oct 9, 2023
Merged

Conversation

georgesittas
Copy link
Collaborator

Motivated by SQLite: https://www.sqlite.org/optoverview.html.

Perhaps we can treat casts as "constant expressions" and do the same for them, e.g.

x = CAST(5 AS BIGINT) AND y = x -- => x = CAST(5 AS BIGINT) AND y = CAST(5 AS BIGINT)
Screenshot 2023-10-08 at 8 45 43 PM

sqlglot/optimizer/simplify.py Show resolved Hide resolved
sqlglot/optimizer/simplify.py Outdated Show resolved Hide resolved
sqlglot/optimizer/simplify.py Outdated Show resolved Hide resolved
and id(column) != column_id
and not (isinstance(parent, exp.Is) and isinstance(parent.expression, exp.Null))
):
column.replace(constant.copy())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is replacement right? That takes it a step further than the sqlite rule, which says it just adds the extra predicate.

Do any planners take advantage of column equality in join conditions?

Copy link
Collaborator Author

@georgesittas georgesittas Oct 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought SQLite does the variable replacement as well, but I'm not 100% sure. Will need to check their source to verify this assumption.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The example doesn’t show replacement. I also wonder if there is different behavior on join conditions.

Copy link
Collaborator Author

@georgesittas georgesittas Oct 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, will dig deeper. Also a good catch, thanks!

Copy link
Collaborator Author

@georgesittas georgesittas Oct 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw I've discovered that there are some gnarly cases that are tricky to take care of - mostly because of coercions (e.g. int to string - see SQLite documentation linked below).

I also wonder if there is different behavior on join conditions.

Interestingly, SQLite doesn't seem to implement this transformation for ON clauses of LEFT, RIGHT joins:

      ...
      /* Do not propagate constants on any ON clause if there is a
      ** RIGHT JOIN anywhere in the query */
      x.mExcludeOn = EP_InnerON | EP_OuterON;
    }else{
      /* Do not propagate constants through the ON clause of a LEFT JOIN */
      x.mExcludeOn = EP_OuterON;
      ...

I'll need to understand their code a bit better to make sure this is the case though..

Some documentation on SQLite's implementation can be found here (screenshot of their source code).

On the other hand, MariaDB mentions this transformation in the context of WHERE clauses, but towards the end of the page they also have this (note the remark about ON expressions):

Screenshot 2023-10-09 at 5 46 56 AM

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding

The example doesn’t show replacement

It's now clear that SQLite does indeed replace the variable with the value:

Screenshot 2023-10-09 at 6 03 43 AM

@georgesittas georgesittas changed the title Feat!(optimizer): propagate constants Feat1(optimizer): propagate constants Oct 9, 2023
@georgesittas georgesittas changed the title Feat1(optimizer): propagate constants Feat(optimizer): propagate constants Oct 9, 2023
@georgesittas
Copy link
Collaborator Author

We discussed about this PR in Slack and decided to keep the constant propagation rule opt-in for now, as it's relatively unstable and needs some more investigation / polishing.

@georgesittas georgesittas merged commit cca58dd into main Oct 9, 2023
@georgesittas georgesittas deleted the jo/propagate_constants branch October 9, 2023 16:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants