Send alerts to prometheus when a workflow fails to be processed.

[khurtado]

Fixes #10215

Status

In development

Description

Send alerts to prometheus when a workflow fails to be processed.

Is it backward compatible (if not, which system it affects?)

YES

[cmsdmwmbot]

Jenkins results:

• Python2 Unit tests: succeeded
• Python3 Unit tests: succeeded
  • 2 changes in unstable tests
• Python2 Pylint check: failed
  • 5 warnings and errors that must be fixed
  • 5 warnings
  • 36 comments to review
• Python3 Pylint check: succeeded
• Pylint py3k check: failed
  • 1 warnings
• Pycodestyle check: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12729/artifact/artifacts/PullRequestReport.html

[cmsdmwmbot]

Jenkins results:

• Python2 Unit tests: succeeded
• Python3 Unit tests: succeeded
  • 2 changes in unstable tests
• Python2 Pylint check: failed
  • 1 warnings and errors that must be fixed
  • 5 warnings
  • 36 comments to review
• Python3 Pylint check: succeeded
• Pylint py3k check: failed
  • 1 warnings
• Pycodestyle check: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12730/artifact/artifacts/PullRequestReport.html

[amaltaro]

Kenyi, these changes are looking good. However, I think these alerts will be better organized if we actually have a wrapper function/method for each of them. We might even want to create a new module called something like MSTransferorAlerts and encapsulate all this information in there.

If you feel like refactoring the notifyLargeData method as well, that would be even better :-D

What do you think?

[khurtado]

@amaltaro : Okay, I encapsulated the alerts in different methods. Could you please check the second commit?

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: succeeded
  • 2 changes in unstable tests
• Python3 Pylint check: succeeded
• Pylint py3k check: succeeded
• Pycodestyle check: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12773/artifact/artifacts/PullRequestReport.html

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: succeeded
  • 4 changes in unstable tests
• Python3 Pylint check: succeeded
• Pylint py3k check: succeeded
• Pycodestyle check: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12774/artifact/artifacts/PullRequestReport.html

[amaltaro]

These changes are looking much better Kenyi! I left a few comments along the code for your consideration though (I think I made single comments instead of a review, sorry!).

[khurtado]

These changes are looking much better Kenyi! I left a few comments along the code for your consideration though (I think I made single comments instead of a review, sorry!).

Alright! Just made the changes.

[amaltaro]

@khurtado this is looking good now. Can you please squash those 3 commits into a single one? Thanks

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: succeeded
  • 3 changes in unstable tests
• Python3 Pylint check: succeeded
• Pylint py3k check: succeeded
• Pycodestyle check: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12787/artifact/artifacts/PullRequestReport.html

[khurtado]

@amaltaro Done!

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: succeeded
  • 1 changes in unstable tests
• Python3 Pylint check: succeeded
• Pylint py3k check: succeeded
• Pycodestyle check: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12788/artifact/artifacts/PullRequestReport.html

[amaltaro]

Thanks Kenyi!