Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add advanced extension field to DdlRel, WriteRel, and UpdateRel #766

Merged
merged 11 commits into from
Jan 29, 2025

Conversation

jcamachor
Copy link
Contributor

No description provided.

@jacques-n
Copy link
Contributor

What are general thoughts on creating field here versus the one already available in RelCommon? RelCommon is kind of like an abstract superclass (if you will) except that protobuf makes you invert the construction.

@jcamachor
Copy link
Contributor Author

Thanks, @jacques-n . I was trying to follow the pattern used for other operators. My assumption was that both fields are included because it allows to have an extension implementation common to all operators (the one under RelCommon) as well as an extension specific to a given operator.

@jacques-n
Copy link
Contributor

Thanks, @jacques-n . I was trying to follow the pattern used for other operators....

Fair. I think we never really figured it out and it shows.

jacques-n
jacques-n previously approved these changes Jan 8, 2025
@jacques-n
Copy link
Contributor

The one problem I struggle with is the in relation location means these are by default semantic changing so a system receiving something at this position must understand or fail the query.

The benefit of using inside hint of relcommon is that an engine can ignore if it likes.

@jcamachor
Copy link
Contributor Author

in relation location means these are by default semantic changing

Isn't it the case that whether these extensions are semantic changing by default depends on whether we use the optimization field or the enhancement field within the AdvancedExtension to define the extension?

@jcamachor jcamachor requested a review from jacques-n January 8, 2025 14:54
@jacques-n
Copy link
Contributor

in relation location means these are by default semantic changing

Isn't it the case that whether these extensions are semantic changing by default depends on whether we use the optimization field or the enhancement field within the AdvancedExtension to define the extension?

lol. yes.

jacques-n
jacques-n previously approved these changes Jan 8, 2025
Copy link
Member

@vbarua vbarua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks reasonable to me. This brings these inline with all the other rels which also have AdvancedExtensions available.

I did leave some comments about the field number we're using. I don't know if it makes sense to reserve fields as opposed to just using the next available one.

proto/substrait/algebra.proto Outdated Show resolved Hide resolved
proto/substrait/algebra.proto Outdated Show resolved Hide resolved
proto/substrait/algebra.proto Outdated Show resolved Hide resolved
@jcamachor
Copy link
Contributor Author

jcamachor commented Jan 9, 2025

I don't know if it makes sense to reserve fields as opposed to just using the next available one.

Thanks, @vbarua . For this, I used the same convention that I observed in extensions defined in other ops. Maybe let's reach consensus with other committers before I change it? @jacques-n , any preferences?

@jcamachor
Copy link
Contributor Author

@vbarua , @jacques-n , how do you think we should proceed here? Thanks

@jcamachor
Copy link
Contributor Author

@vbarua , @jacques-n , how do you think we should proceed here? Thanks

I have updated the field numbers.

@jacques-n
Copy link
Contributor

I agree with @vbarua that we should just use next available. I didn't notice that on my review. It looks like we're still using not-next?

@jcamachor
Copy link
Contributor Author

In the most recent commit, I changed it to use next available value.

@jcamachor
Copy link
Contributor Author

@jacques-n , @vbarua , all feedback has been addressed, could we merge this change? (I believe current check failure is unrelated to this change but I'm not super familiar with it.) Thanks

@jacques-n jacques-n merged commit a428f96 into substrait-io:main Jan 29, 2025
13 checks passed
@jcamachor jcamachor deleted the extensions branch January 29, 2025 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants