Assign the correct activity for Tape_Test RSEs #11067

jhonatanamado · 2022-04-05T17:05:37Z

Status

Tested with a replay

Description

Add the proper activity for the T0-Agent when is subscribing data to Tape_Test rses. Small changes on how this code recognize a Tape_Test RSE. With this changes, the rule replica subscribed to T0_CH_CERN_Tape_Test gives the following:

rucio rule-info 0035c69b588045769195f241df939406
Id:                         0035c69b588045769195f241df939406
Account:                    tier0_replay
Scope:                      cms
Name:                       /ExpressCosmics/Tier0_REPLAY_2022-v98/RAW
RSE Expression:             T0_CH_CERN_Tape_Test
Copies:                     1
State:                      REPLICATING
Locks OK/REPLICATING/STUCK: 0/1/0
Grouping:                   ALL
Expires at:                 None
Locked:                     True
Weight:                     None
Created at:                 2022-04-05 03:33:03
Updated at:                 2022-04-05 03:33:55
Error:                      None
Subscription Id:            None
Source replica expression:  None
Activity:                   T0 Tape
Comment:                    T0 WMAgent automatic container rule
Ignore Quota:               False
Ignore Availability:        False
Purge replicas:             False
Notification:               NO
End of life:                None
Child Rule Id:              None

Is it backward compatible (if not, which system it affects?)

Maybe

Related PRs

None

External dependencies / deployment changes

None

cmsdmwmbot · 2022-04-05T17:19:53Z

Jenkins results:

Python3 Unit tests: succeeded
- 7 tests added
- 3 changes in unstable tests
Python3 Pylint check: failed
- 12 warnings and errors that must be fixed
- 6 warnings
- 11 comments to review
Pylint py3k check: succeeded
Pycodestyle check: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12986/artifact/artifacts/PullRequestReport.html

amaltaro

Thanks for providing this patch, Jhonatan. Please see a couple of comments along the code.

amaltaro · 2022-04-06T04:20:05Z

src/python/WMComponent/RucioInjector/RucioInjectorPoller.py

@@ -465,7 +465,7 @@ def insertContainerRules(self):
                              grouping="ALL",
                              comment=ruleComment,
                              meta=self.metaData)
-            if not rseName.endswith("_Tape"):
+            if "Tape" not in rseName:


This is a risky check, and given that we never produce data at a Tape endpoint, I would suggest to revert this change.

Agreed, the way how this check has been implemented originally is better.

amaltaro · 2022-04-06T04:22:44Z

src/python/WMComponent/RucioInjector/RucioInjectorPoller.py

@@ -529,7 +529,7 @@ def _activityMap(self, rseName):
        """
        if not self.isT0agent and not rseName.endswith("_Tape"):
            return "Production Output"
-        elif self.isT0agent and rseName.endswith("_Tape"):
+        elif self.isT0agent and "Tape" in rseName:


I would suggest a check like:

elif self.isT0agent and (rseName.endswith("_Tape") or rseName.endswith("_Tape_Test"):

People are very creative and I wouldn't be surprised to, one day, see a Disk endpoint with Tape in its name :-D

I can second Alan's suggestion here.

todor-ivanov

Thanks Jhonatan
I have basically agreed with Alan's suggestions here too. Please take a look the comments inline.

todor-ivanov · 2022-04-07T12:26:52Z

src/python/WMComponent/RucioInjector/RucioInjectorPoller.py

@@ -529,7 +529,7 @@ def _activityMap(self, rseName):
        """
        if not self.isT0agent and not rseName.endswith("_Tape"):
            return "Production Output"
-        elif self.isT0agent and rseName.endswith("_Tape"):
+        elif self.isT0agent and "Tape" in rseName:


I can second Alan's suggestion here.

todor-ivanov · 2022-04-07T12:28:25Z

src/python/WMComponent/RucioInjector/RucioInjectorPoller.py

@@ -465,7 +465,7 @@ def insertContainerRules(self):
                              grouping="ALL",
                              comment=ruleComment,
                              meta=self.metaData)
-            if not rseName.endswith("_Tape"):
+            if "Tape" not in rseName:


Agreed, the way how this check has been implemented originally is better.

cmsdmwmbot · 2022-04-07T23:22:29Z

Jenkins results:

Python3 Unit tests: succeeded
- 7 tests added
Python3 Pylint check: failed
- 12 warnings and errors that must be fixed
- 6 warnings
- 11 comments to review
Pylint py3k check: succeeded
Pycodestyle check: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12993/artifact/artifacts/PullRequestReport.html

cmsdmwmbot · 2022-04-09T05:26:33Z

Jenkins results:

Python3 Unit tests: failed
- 1 new failures
- 7 tests added
- 1 changes in unstable tests
Python3 Pylint check: failed
- 12 warnings and errors that must be fixed
- 6 warnings
- 11 comments to review
Pylint py3k check: succeeded
Pycodestyle check: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12996/artifact/artifacts/PullRequestReport.html

Add the proper activity for the T0-Agent when is subscribing data to Tape_Test rses.

cmsdmwmbot · 2022-04-09T05:42:23Z

Jenkins results:

Python3 Unit tests: succeeded
- 7 tests added
- 1 changes in unstable tests
Python3 Pylint check: failed
- 12 warnings and errors that must be fixed
- 6 warnings
- 11 comments to review
Pylint py3k check: succeeded
Pycodestyle check: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12997/artifact/artifacts/PullRequestReport.html

todor-ivanov

Thanks @jhonatanamado
looks good to me

amaltaro

@jhonatanamado this is even a better implementation than the one suggested. I wasn't aware we could check it against a tuple! Thanks

amaltaro requested review from todor-ivanov and amaltaro April 6, 2022 01:47

amaltaro requested changes Apr 6, 2022

View reviewed changes

todor-ivanov requested changes Apr 7, 2022

View reviewed changes

jhonatanamado force-pushed the fix_T0Activity branch from c3aa6b6 to 45b5a89 Compare April 7, 2022 23:10

jhonatanamado added the PR: Do not merge yet label Apr 8, 2022

jhonatanamado force-pushed the fix_T0Activity branch from 45b5a89 to 60c9e7e Compare April 9, 2022 05:10

Assign the correct activity for Tape_Test RSEs

bb8cd78

Add the proper activity for the T0-Agent when is subscribing data to Tape_Test rses.

jhonatanamado force-pushed the fix_T0Activity branch from 60c9e7e to bb8cd78 Compare April 9, 2022 05:32

jhonatanamado requested review from amaltaro and todor-ivanov April 9, 2022 05:54

jhonatanamado removed the PR: Do not merge yet label Apr 9, 2022

todor-ivanov approved these changes Apr 11, 2022

View reviewed changes

amaltaro approved these changes Apr 12, 2022

View reviewed changes

amaltaro added the PR: Need wmagent branch label Apr 12, 2022

amaltaro merged commit ce7fd8b into dmwm:master Apr 12, 2022

jhonatanamado deleted the fix_T0Activity branch April 12, 2022 12:43

amaltaro mentioned this pull request Apr 12, 2022

Bundle of patches for WMAgent 2.0.2.patch1 #11090

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Assign the correct activity for Tape_Test RSEs #11067

Assign the correct activity for Tape_Test RSEs #11067

jhonatanamado commented Apr 5, 2022

cmsdmwmbot commented Apr 5, 2022

amaltaro left a comment

amaltaro Apr 6, 2022

todor-ivanov Apr 7, 2022

amaltaro Apr 6, 2022

todor-ivanov Apr 7, 2022

todor-ivanov left a comment

todor-ivanov Apr 7, 2022

todor-ivanov Apr 7, 2022

cmsdmwmbot commented Apr 7, 2022

cmsdmwmbot commented Apr 9, 2022

cmsdmwmbot commented Apr 9, 2022

todor-ivanov left a comment

amaltaro left a comment

Assign the correct activity for Tape_Test RSEs #11067

Assign the correct activity for Tape_Test RSEs #11067

Conversation

jhonatanamado commented Apr 5, 2022

Status

Description

Is it backward compatible (if not, which system it affects?)

Related PRs

External dependencies / deployment changes

cmsdmwmbot commented Apr 5, 2022

amaltaro left a comment

Choose a reason for hiding this comment

amaltaro Apr 6, 2022

Choose a reason for hiding this comment

todor-ivanov Apr 7, 2022

Choose a reason for hiding this comment

amaltaro Apr 6, 2022

Choose a reason for hiding this comment

todor-ivanov Apr 7, 2022

Choose a reason for hiding this comment

todor-ivanov left a comment

Choose a reason for hiding this comment

todor-ivanov Apr 7, 2022

Choose a reason for hiding this comment

todor-ivanov Apr 7, 2022

Choose a reason for hiding this comment

cmsdmwmbot commented Apr 7, 2022

cmsdmwmbot commented Apr 9, 2022

cmsdmwmbot commented Apr 9, 2022

todor-ivanov left a comment

Choose a reason for hiding this comment

amaltaro left a comment

Choose a reason for hiding this comment