Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Automatic restart of streaming applications and optimization of power problems #22

Closed
2 tasks done
wxyn opened this issue May 27, 2022 · 0 comments · Fixed by #29
Closed
2 tasks done
Labels
enhancement New feature or request

Comments

@wxyn
Copy link

wxyn commented May 27, 2022

Search before asking

  • I had searched in the issues and found no similar feature requirement.

Problem Description

Streamis tasks can fail and terminate due to various environmental issues (especially Streamis Flink tasks that currently do not support automatic HA pull-up), so it is important to recognize that the Streamis task automatically restarts if it fails abnormally.
Streamis的任务可能会因为各种环境问题发生失败终止的情况(特别是目前Streamis的Flink任务不支持HA自动拉起的情况),所以要识别出Streamis任务异常终止后程序自动重启改任务。

Description

  1. Try to obtain the checkpoint information. If no checkpoint information is available, check whether the checkpoint policy is configured. If no checkpoint policy is configured, ignore it.
  2. Add the checkpoint information as a task parameter to the task and start the task asynchronously.
    Note: Task start jobs are unique, so restart/start operations are idempotent and only one job can start at a time
    1)先尝试拿到checkpoint信息,如果没有checkpoint信息则查看任务是否有配置checkpoint策略,如果没有则忽略。
    2)将checkpoint信息作为task参数放到task信息里,异步启动task任务。
    注:任务启动作业是惟一的,所以要求重启/启动操作的幂等性,同一时间内只能有一个作业启动

Use case

No response

solutions

No response

Anything else

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant