Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Requirements: upgrade Celery #10505

Open
humitos opened this issue Jul 3, 2023 · 0 comments
Open

Requirements: upgrade Celery #10505

humitos opened this issue Jul 3, 2023 · 0 comments
Labels
Accepted Accepted issue on our roadmap Improvement Minor improvement to code

Comments

@humitos
Copy link
Member

humitos commented Jul 3, 2023

I added this note in #10503

# NOTE: we have to pin to this version because Celery>=5.3.x detects our ``ConfigError`` exception is not picklable
# and creates a ``UnpickleableExceptionWrapper`` which is not what our code expects.
# This may be due to a Python bug which can't pickle/unpickle the exception properly.
# We could consider either:
#  - handle ``UnpickleableExceptionWrapper`` in our code and inspect what's the inner exception
#  - find a way to disable pickling the exceptions
#  - make our custom exception picklable
#
# References:
#  - https://github.com/celery/celery/pull/8149
#  - https://github.com/celery/celery/blob/2b4b500ca1212016824a5fa2996cfb752f0763a7/celery/utils/serialization.py#L154
#  - https://github.com/python/cpython/issues/76877
#
# Reproducible example:
# >>> import pickle
# >>> class MyException(Exception):
# ...   def __init__(self, a, b):
# ...     self.a = a
# ...     self.b = b
# ...     super().__init__(a)
# ...
# >>> pickle.loads(pickle.dumps(MyException("a", "b")))
# Traceback (most recent call last):
#   File "<stdin>", line 1, in <module>
# TypeError: MyException.__init__() missing 1 required positional argument: 'b'
# >>>
celery==5.2.7

We need to do some work here to be able to upgrade Celery to 5.3.x

@humitos humitos added Improvement Minor improvement to code Accepted Accepted issue on our roadmap labels Jul 3, 2023
@github-project-automation github-project-automation bot moved this to Planned in 📍Roadmap Jul 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Accepted Accepted issue on our roadmap Improvement Minor improvement to code
Projects
Status: Planned
Development

No branches or pull requests

1 participant