-
Notifications
You must be signed in to change notification settings - Fork 773
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feat(optimizer): propagate constants #2386
Conversation
and id(column) != column_id | ||
and not (isinstance(parent, exp.Is) and isinstance(parent.expression, exp.Null)) | ||
): | ||
column.replace(constant.copy()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is replacement right? That takes it a step further than the sqlite rule, which says it just adds the extra predicate.
Do any planners take advantage of column equality in join conditions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought SQLite does the variable replacement as well, but I'm not 100% sure. Will need to check their source to verify this assumption.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The example doesn’t show replacement. I also wonder if there is different behavior on join conditions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, will dig deeper. Also a good catch, thanks!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Btw I've discovered that there are some gnarly cases that are tricky to take care of - mostly because of coercions (e.g. int to string - see SQLite documentation linked below).
I also wonder if there is different behavior on join conditions.
Interestingly, SQLite doesn't seem to implement this transformation for ON
clauses of LEFT
, RIGHT
joins:
...
/* Do not propagate constants on any ON clause if there is a
** RIGHT JOIN anywhere in the query */
x.mExcludeOn = EP_InnerON | EP_OuterON;
}else{
/* Do not propagate constants through the ON clause of a LEFT JOIN */
x.mExcludeOn = EP_OuterON;
...
I'll need to understand their code a bit better to make sure this is the case though..
Some documentation on SQLite's implementation can be found here (screenshot of their source code).
On the other hand, MariaDB mentions this transformation in the context of WHERE
clauses, but towards the end of the page they also have this (note the remark about ON
expressions):
![Screenshot 2023-10-09 at 5 46 56 AM](https://private-user-images.githubusercontent.com/46752250/273489646-945f3f47-1f3e-4805-aed5-8aa1b2575efb.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mzk1MzA1NzMsIm5iZiI6MTczOTUzMDI3MywicGF0aCI6Ii80Njc1MjI1MC8yNzM0ODk2NDYtOTQ1ZjNmNDctMWYzZS00ODA1LWFlZDUtOGFhMWIyNTc1ZWZiLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMTQlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjE0VDEwNTExM1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTg2M2Q1YjJkMTczZmJjYmJkMzcwMWZjZjMzMGM2MjQ2NjRjNjQwZDMxZDBmYWYwODE0ODcyNzI0NzQ2ODhmNjUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.LCws26Kia1yiFKVgSWHYJViYo6v3HbVDbdwO5vHURlE)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We discussed about this PR in Slack and decided to keep the constant propagation rule opt-in for now, as it's relatively unstable and needs some more investigation / polishing. |
Motivated by SQLite: https://www.sqlite.org/optoverview.html.
Perhaps we can treat casts as "constant expressions" and do the same for them, e.g.