Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow periodic tasks to define their exact wall-clock time to run #48

Open
juledwar opened this issue Jun 3, 2024 · 2 comments
Open

Comments

@juledwar
Copy link
Contributor

juledwar commented Jun 3, 2024

The periodics are currently effectively a 'time since it last ran, which in turn is time since it first ever ran'.

It would be useful for periodics to define a wall clock start and offset, for a precise execution time (assuming free workers are available).

Tasks are currently defined like:

    def task(self, func: Optional[Callable]=None, name: Optional[str]=None,
             queue: Optional[str]=None, max_retries: Optional[Number]=None,
             periodicity: Optional[timedelta]=None,
             max_concurrency: Optional[int]=None):
        """Decorator to register a task function.

I propose that we extend the parameters to accept a periodicity_start, another datetime.timedelta object that is used to state the offset from the next largest unit. For example:

  • periodicity_start=timedelta(minutes=1), periodicity=timedelta(minutes=5) would tell the code to run the task every 5 minutes, starting from 1 minute past the hour i.e. 1m, 6m, 11m past the hour, etc.
  • periodicity_start=timedelta(hours=3), periodicity=timedelta(hours=6) would tell the code to run the task every 6 hours at 3am, 9am, 3pm and 9pm.
@juledwar
Copy link
Contributor Author

juledwar commented Jun 3, 2024

This should be simple to implement: the register_periodic_tasks can change the "now" value to the required starting offset as per the task definition.

The hard part is making sure the parameters make logical sense and rejecting bad ones.

@juledwar
Copy link
Contributor Author

juledwar commented Jun 3, 2024

Additionally, any existing tasks already in Redis that are subsequently changed to include a periodicity_start would need updating in Redis. In practice, this means going over all of them on every worker startup to make sure they are defined correctly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant