Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Evaluate None in SQLAlchemy's extended JSON type decorator #45119

Merged

Conversation

shahar1
Copy link
Contributor

@shahar1 shahar1 commented Dec 20, 2024

closes: #22245

As of today, at least in mapped operators for map_index > 0, the next_kwargs column is saved as the string null in the DB, instead of a real <null>. That is unlike mapped operators with indeces -1 or 0, as well as unmapped operators.
As far as I'm concerned, this inconsistency doesn't affect any logic that depends on it, since it is still deserialized as Python's None - but it will be nice to save a few bytes and ensure consistency.
While I haven't figured out why it happens only in that case - I've found that applying the flag should_evaluate_none = True seems to solve it. I didn't find a proper way to test it, since it is always deserialized as Python's None.

Mapped operators are yet to be implemented in the main with the new task sdk, but if tests won't indicate that it breaks anything - I think that it will be ok to merge.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@shahar1
Copy link
Contributor Author

shahar1 commented Dec 21, 2024

@potiuk are we ok with merging this PR on main?
You approved and merged the backport, just wondered if you just missed this one or is it on purpose :)

@potiuk potiuk merged commit ad9bbf9 into apache:main Dec 21, 2024
49 checks passed
@potiuk
Copy link
Member

potiuk commented Dec 21, 2024

Ups

@potiuk
Copy link
Member

potiuk commented Dec 21, 2024

Thanks 😓 .... Not sure how I missed it :)

@shahar1 shahar1 deleted the evaluate-none-in-extended-json-type-decorator branch December 21, 2024 20:01
LefterisXefteris pushed a commit to LefterisXefteris/airflow that referenced this pull request Jan 5, 2025
agupta01 pushed a commit to agupta01/airflow that referenced this pull request Jan 6, 2025
@utkarsharma2 utkarsharma2 added the type:bug-fix Changelog: Bug Fixes label Jan 28, 2025
got686-yandex pushed a commit to got686-yandex/airflow that referenced this pull request Jan 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:bug-fix Changelog: Bug Fixes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

next_kwargs value sometimes value NULL sometimes string null
3 participants