Swap NotMasterError with NotPrimaryError.

[todor-ivanov]

Fixes #11012

Status

Ready

Description

The current PR is intended to handle the newly introduced exception NotPrimaryError on the pymongo client side, which is about to substitute the old one NotMasterError, starting with pymongo version 4.01 . We have tested and proved that on the server side the latest version is already supporting both the old and the new clients, so no errors in exception handling are originating from client vs server version mismatch.

Is it backward compatible (if not, which system it affects?)

NO

Related PRs

There is about to be a cmsdist related PR to upgrade the pymongo rpm package:
cms-sw/cmsdist#7654

External dependencies / deployment changes

Yes.

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: succeeded
  • 7 tests deleted
  • 3 changes in unstable tests
• Python3 Pylint check: succeeded
• Pylint py3k check: succeeded
• Pycodestyle check: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12831/artifact/artifacts/PullRequestReport.html

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: succeeded
  • 7 tests deleted
• Python3 Pylint check: succeeded
• Pylint py3k check: succeeded
• Pycodestyle check: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12832/artifact/artifacts/PullRequestReport.html

[amaltaro]

Todor, don't we have to update any of the exception handling in our baseline source code? I had the impression it required a pymongo bump and a WMCore change as well.

[todor-ivanov]

Hi @amaltaro This
https://github.com/dmwm/WMCore/pull/11016/files#diff-cd601ffcba537b986d36fc945d0d34012052b4c198c3fbd14335e01579849b85R141

is the only strictly WMCore related change, which may affect other services using Mongodb. It is related to a change of a deprecated method in pymongo 4.0.1. But this one is there since a while now (with a deprecation warning). The new method (client.list_database_names()) exists even in pymongo 3.10.1, see [1]. So this change is not going to break anything in other services using pymongo. The rest of the code changes affect only MSUnmerged micro service.

[1]

In [2]: msUnmergedDB.command({'buildInfo':1})['version']
Out[2]: '5.0.3'

In [3]: import pymongo

In [4]: pymongo.version
Out[4]: '3.10.1'

In [5]: msUnmergedDB.client.database_names()
~/WMCore.venv3/bin/ipython:1: DeprecationWarning: database_names is deprecated. Use list_database_names instead.
  #!~/WMCore.venv3/bin/python3
Out[5]: ['admin', 'config', 'local', 'msUnmergedDBPreProd', 'msUnmergedDBProd']

In [6]: msUnmergedDB.client.list_database_names()
Out[6]: ['admin', 'config', 'local', 'msUnmergedDBPreProd', 'msUnmergedDBProd']

[amaltaro]

Todor, it looks like I forgot to scroll down and missed those baseline changes.
While these changes look good to me, I would say it cannot be merged until we cover all the 3 MongoDB use cases that we have in WMCore (as mentioned a minute ago in the GH issue itself).

[todor-ivanov]

Thank you @vkuznet !

BTW I think I have somehow closed that issue by mistake. Reopening it with the current comment.

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: succeeded
  • 7 tests deleted
• Python3 Pylint check: succeeded
• Pylint py3k check: succeeded
• Pycodestyle check: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12835/artifact/artifacts/PullRequestReport.html

[amaltaro]

Just making sure my review - and discussion - gets posted to the PR instead of the issue itself: #11012 (comment)

I think it would be important to have a fully functional change. And given that it was mentioned it would be a near future issue, we might want to decide to come back to this in the very near future as well (hopefully with the VM setup no longer valid).

[amaltaro]

Okay, I tried to upgrade MongoDB in COMP, but it's not that straight forward.

Let's get this merged then, but before that, we should check why those unit tests are reported as deleted (I think we need to first upgrade pymongo version, I will get that cmsdist PR merged and then we can come back to this later today or tomorrow).

[amaltaro]

test this please

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: succeeded
  • 1 changes in unstable tests
• Python3 Pylint check: succeeded
• Pylint py3k check: succeeded
• Pycodestyle check: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12846/artifact/artifacts/PullRequestReport.html

[amaltaro]

This looks better now.

Note that this will likely break our MSOutput and MSUnmerged services when running on RPM-based MongoDB setup. @todor-ivanov can you please update the initial description with this note as well? Thanks