Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Properly validate RequestPriority parameter #9994

Merged
merged 2 commits into from
Oct 16, 2020
Merged

Conversation

amaltaro
Copy link
Contributor

@amaltaro amaltaro commented Oct 15, 2020

Fixes #9992

Status

ready

Description

Make sure the RequestPriority spec parameter is properly validated, regardless of which state the workflow is.
For clarity: it must be of the integer type and be in the range of [0, 1e6]

Is it backward compatible (if not, which system it affects?)

Yes (well, we need to chase those requests with float value, otherwise an ACDC might fail to be created...)

Related PRs

none

External dependencies / deployment changes

none

@cmsdmwmbot
Copy link

Jenkins results:

  • Unit tests: succeeded
    • 49 tests no longer failing
    • 1 tests added
    • 5 changes in unstable tests
  • Pylint check: failed
    • 2 warnings and errors that must be fixed
    • 16 warnings
    • 150 comments to review
  • Pycodestyle check: succeeded
    • 4 comments to review
  • Python3 compatibility checks: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/10527/artifact/artifacts/PullRequestReport.html

@cmsdmwmbot
Copy link

Jenkins results:

  • Unit tests: succeeded
    • 1 tests added
  • Pylint check: failed
    • 2 warnings and errors that must be fixed
    • 16 warnings
    • 150 comments to review
  • Pycodestyle check: succeeded
    • 4 comments to review
  • Python3 compatibility checks: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/10531/artifact/artifacts/PullRequestReport.html

@amaltaro
Copy link
Contributor Author

Tests went fine!

Copy link
Contributor

@todor-ivanov todor-ivanov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amaltaro I made one comment in the code. Please take a look at it. Thanks!

reqPrioDefin = StdBase.getWorkloadCreateArgs()['RequestPriority']
try:
# Web GUI sends it as a string
reqArgs['RequestPriority'] = int(reqArgs['RequestPriority'])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't that be more clear if one just does the following here:

if not isinstance(reqArgs['RequestPriority'], reqPrioDefin['type']):
    raise InvalidSpecParameterValue(msg)

We need to check for the type only and not for the possibility to cast the object to a given type, because we never do so down the code. And we will also avoid hardcoding a int type check in the middle of the code while we have the object type defined from StdBase. I think we should take it from the upper level object otherwise we risk to have objects that silently transform from one type to another, which in my opinion is a bug.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I meant to write:

reqArgs['RequestPriority'] = reqPrioDefin['type'](reqArgs['RequestPriority'])

About simply checking the data type, it would be easier if we were already running this software in python3. Since we are still in py2, any check for string is just another complication in the near future.

@cmsdmwmbot
Copy link

Jenkins results:

  • Unit tests: succeeded
    • 1 tests added
  • Pylint check: failed
    • 2 warnings and errors that must be fixed
    • 16 warnings
    • 150 comments to review
  • Pycodestyle check: succeeded
    • 4 comments to review
  • Python3 compatibility checks: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/10534/artifact/artifacts/PullRequestReport.html

Copy link
Contributor

@todor-ivanov todor-ivanov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me
Thanks @amaltaro

@amaltaro amaltaro force-pushed the fix-9992 branch 2 times, most recently from edaa43f to 8a5d174 Compare October 16, 2020 14:18
improve error message

cast using type from stdbase

use type check

remove blank space

fix message
fix unit tests
@cmsdmwmbot
Copy link

Jenkins results:

  • Unit tests: failed
    • 1 tests added
  • Pylint check: failed
    • 2 warnings and errors that must be fixed
    • 16 warnings
    • 150 comments to review
  • Pycodestyle check: succeeded
    • 4 comments to review
  • Python3 compatibility checks: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/10535/artifact/artifacts/PullRequestReport.html

@cmsdmwmbot
Copy link

Jenkins results:

  • Unit tests: succeeded
    • 1 tests added
  • Pylint check: failed
    • 2 warnings and errors that must be fixed
    • 16 warnings
    • 150 comments to review
  • Pycodestyle check: succeeded
    • 4 comments to review
  • Python3 compatibility checks: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/10536/artifact/artifacts/PullRequestReport.html

@cmsdmwmbot
Copy link

Jenkins results:

  • Unit tests: succeeded
    • 1 tests added
  • Pylint check: failed
    • 2 warnings and errors that must be fixed
    • 16 warnings
    • 150 comments to review
  • Pycodestyle check: succeeded
    • 4 comments to review
  • Python3 compatibility checks: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/10537/artifact/artifacts/PullRequestReport.html

@amaltaro
Copy link
Contributor Author

@todor-ivanov sorry for bugging you again here. I didn't manage to reproduce the string type on my VM, neither via script, nor via Gui. So I made the change to simply check for the data type. Can you please have another look? Thanks

@amaltaro amaltaro requested a review from todor-ivanov October 16, 2020 14:33
Copy link
Contributor

@todor-ivanov todor-ivanov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks perfect to me. Thanks!

@amaltaro
Copy link
Contributor Author

Thanks!

@amaltaro amaltaro merged commit 5ff432f into dmwm:master Oct 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RequestPriority validation only works during the workflow creation
3 participants