[py2py3] Migration at level scr/python/A/B/C - slice 26

[mapellidario]

Intro

Our purpose is to continue with complete the gradual migration.
This is the 26-th and last slice in which the migration has been divided into.
We have split migration units at level src/python/A/B/C.

Unless the plan is changed The plan did not change, so this is the last slice in which the migration is split into! 🎉

Impact of the new feature

I run pylint --py3k -d W1618 on whole dmwm/WMCore and I will paste here its report, so that we have a todo list with the last things required

• src/ [1]
• test/ [2]
• setup_*.py, bin/, tools/, standards/, etc/, doc/, deploy/ are ok
• there are no other issues related to wrong character to stop iteration when reading a file, such as

- with open(filename, 'rb') as fd: contents = iter(lambda: fd.read(blockSize), '')
+ with open(filename, 'rb') as fd: contents = iter(lambda: fd.read(blockSize), b'')

Keep in mind

• i suppressed W1618, because it complains about absolute imports, which is not a problem for us now.

Is your feature request related to a problem? Please describe.

The end goal is to have WMCore py2 and py3 compatible.

Describe the solution you'd like

We should fix the report from pylint

other info

[1]

• Module src.python.WMQuality.Emulators.PhEDExClient.MockPhEDExApi. deprecated
• Module src.python.Utils.Timers. not critical use of round.
• Module src.python.WMCore.Configuration src/python/WMCore/Configuration.py:74:0: W1641: Implementing __eq__ without also implementing __hash__ (eq-without-hash). [3]
• Module src.python.WMCore.Configuration src/python/WMCore/Configuration.py:143:33: W1612: unicode built-in referenced (unicode-builtin). protected by if not PY3
• Module src.python.WMCore.Services.LogDB.LogDBBackend: not critical use of round
• Module src.python.WMCore.Services.PhEDEx.PhEDEx. deprecated
• Module src.python.WMCore.Services.PhEDEx.DataStructs.SubscriptionList. deprecaeted
• Module src.python.WMCore.WorkerThreads.BaseWorkerThread. fixed
• Module src.python.WMCore.WMSpec.StdSpecs.StdBase. investigate in [py2py3] effect of py3 round, mostly on job splitting #10354
• Module src.python.WMCore.REST.Error. not a problem. we implemented what pylint says its missing.
• Module src.python.WMCore.ReqMgr.Service.Request. not a problem, we check if what pylint complains for not being present in py3 is actually present
• Module src.python.WMCore.ReqMgr.Service.RequestAdditionalInfo. same as previous
• Module src.python.WMCore.MicroService.Tools.Common. non critical use of round
• Module src.python.WMCore.WMRuntime.Unpacker. tested. works in py3
• Module src.python.WMCore.JobSplitting.EventAwareLumiBased. see [py2py3] effect of py3 round, mostly on job splitting #10354
• Module src.python.WMCore.JobSplitting.LumiBased. see [py2py3] effect of py3 round, mostly on job splitting #10354
• Module src.python.WMCore.JobSplitting.FileBased. see [py2py3] effect of py3 round, mostly on job splitting #10354
• Module src.python.WMCore.JobSplitting.EventAwareLumiByWork. see [py2py3] effect of py3 round, mostly on job splitting #10354
• Module src.python.WMCore.DataStructs.MathStructs.ContinuousSummaryHistogram. non critical use of round

[2]

• Module test.data.ReqMgr.validate-test-wfs. protected by a try on import, basically equivalent to if PY2
• Module test.python.WMCore_t.WMException_py2_t. this is supposed not to work in py3
• Module test.python.WMCore_t.BossAir_t.MockPlugin_t. fixed, division
• Module test.python.WMCore_t.Misc_t.WMAgent_t. fixed, division
• Module test.python.WMCore_t.WMBS_t.JobSplitting_t.FixedDelay_t. fixed division
• Module test.python.WMCore_t.WMBS_t.JobSplitting_t.LumiBased_t. see [py2py3] effect of py3 round, mostly on job splitting #10354
• Module test.python.WMCore_t.WMBS_t.JobSplitting_t.RunBased_t. fixed, division
• Module test.python.WMCore_t.Agent_t.Heartbeat_t. non critical use of round
• Module test.python.WMCore_t.REST_t.Api_t. see Fix Unittests in Api_t.Tester #10757
• Module test.python.WMCore_t.REST_t.Daemon_t. fixed range and iteritems.
• Module test.python.WMCore_t.Services_t.PhEDEx_t.PhEDEx_t. deprecated
• Module test.python.WMComponent_t.DBS3Buffer_t.DBSUpload_t. fixed, division

[3]

Module src.python.WMCore.Configuration src/python/WMCore/Configuration.py:74:0: W1641: Implementing __eq__ without also implementing __hash__ (eq-without-hash).

• the meaning of this checker is explained at Add a Python 3 warning for implementing __eq__ without also implentnting __hash__ pylint-dev/pylint#1115 and at https://gist.github.com/rowillia/c0feed97c1863b2d8e5a3ed73712df65

Py2 tries to be smart and uses object.__hash__ by default if the user does not define __hash__ in his class. This makes classes with potential mutable attributes to be hashable, which is not what we want. Python3 returns a TypeError: unhashable type as soon as possible.

class JustEq(object):
   def __init__(self, x):
     self.x = x

   def __eq__(self, other):
     return self.x == other.x

list_of_things = [JustEq(1), JustEq(2), JustEq(3)]
set_of_things = set(list_of_things)  # py3: fails with `TypeError: unhashable type: 'JustEq'`

print(JustEq(1) in list_of_things)   # py2: prints 'True'
print(JustEq(1) in set_of_things)    # py2: prints 'False'.  Wat.

With this is mind, I would not change Configuration.py:ConfigSection. I do not expect dmwm/WMCore to contain bugs related to this, since i am confident that we would have noticed such a strange behavior in all these years. However, in the remote possibility that we have such a bug, if we do not change anything, then py3 will return a TypeError when we try to hash an unhashable type, which is all we want.

References:

• https://eng.lyft.com/hashing-and-equality-in-python-2ea8c738fb9d
• https://hynek.me/articles/hashes-and-equality/