TW still fails in py3 inside WMCore/Credential

[belforte]

I do not know why this did not surface earlier... anyhow looks pretty "standard"

  File "/data/srv/TaskManager/py3.211213-b444c6ef2cda45af29ca0fe543718abe/slc7_amd64_gcc630/cms/crabtaskworker/py3.211213-b444c6ef2cda45af29ca0fe543718abe/lib/python3.8/site-packages/TaskWorker/Actions/MyProxyLogon.py", line 80, in execute
    (userproxy, usergroups) = self.tryProxyLogon(proxycfg=proxycfg)
  File "/data/srv/TaskManager/py3.211213-b444c6ef2cda45af29ca0fe543718abe/slc7_amd64_gcc630/cms/crabtaskworker/py3.211213-b444c6ef2cda45af29ca0fe543718abe/lib/python3.8/site-packages/TaskWorker/Actions/MyProxyLogon.py", line 35, in tryProxyLogon
    userproxy = proxy.getProxyFilename(serverRenewer=True)  # this only returns a filename
  File "/data/srv/TaskManager/py3.211213-b444c6ef2cda45af29ca0fe543718abe/slc7_amd64_gcc630/cms/crabtaskworker/py3.211213-b444c6ef2cda45af29ca0fe543718abe/lib/python3.8/site-packages/WMCore/Credential/Proxy.py", line 263, in getProxyFilename
    proxyFilename = os.path.join(self.credServerPath, sha1(
TypeError: Unicode-objects must be encoded before hashing

[amaltaro]

dmwm/WMCore#10912
?

[belforte]

indeed I had fixed it in my private fork
belforte/WMCore@1be8e3c
I guess I failed to communicate this to @mapellidario in order to be added to the branch which we use now in the build. Sorry Dario, another build is needed.

[belforte]

yes Alan. Sorry, my memory is so bad :-( Anyhow, just a matter of "being thorough and careful"

[belforte]

@mapellidario I have switched crab-dev-tw01 to run from GH using my WMCore fork above and CRAB py3.21213 and it is OK now. (yeah.. I have also hacked cmsweb-test1 a bit slipping my personal proxy in to avoid the expired hostcert... good for 7 days or next pod restart, whichever comes first).

[belforte]

Dario, I assign this to you as a reminder. You can close once you have a build with the fix.

[belforte]

spec file with a working build was created via cms-sw/cmsdist#7502
@mapellidario can we close ? Or do you want to keep it open to remind us what to check when we'll want to switch back to to dmwm/WMCore repository ?
There are a few things which now you have fixed only in your branch besides this, maybe better to keep an ad hoc issue ?

[mapellidario]

I think that we can close this issue.

in our private branch at the moment we have three changes:

• (TW) same commit as in [py3] CredentialProxy - convert to bytes the input of hashlib.sha1() WMCore#10918, which has been merged by Alan. If we move back to building TW with dmwm/WMCore at their next release, then we will be fine.
• (REST) same commit as [py3] REST/Server.py - encode to bytes input of hashlib.sha1 WMCore#10917, which has been merged, so If we move back to building crabserver with dmwm/WMCore at their next release, then we will be fine.
• (REST) mapellidario/WMCore@23fef9b which raises an exception when we have unexpected types being inserted into the DB. I would keep this change into our crabserver build for as long as reasonably possible.

Therefore

• crabtaskworker.spec: we can switch to the new dmwm/WMCore version, when they release a new one.
• crabserver.spec : I would continue using my own private branch

[belforte]

thanks Dario, What about point 4 still open in #6830 ?
I think you mentioned that it was something in WMCore, can you add some comment there ?