Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade sqlalchemy in our python3 cmsdist stack #11205

Closed
amaltaro opened this issue Jul 6, 2022 · 1 comment · Fixed by #11257
Closed

Upgrade sqlalchemy in our python3 cmsdist stack #11205

amaltaro opened this issue Jul 6, 2022 · 1 comment · Fixed by #11257
Assignees
Labels
Enhancement Medium Priority Technical Debt Used to track issues that address technical needs internal to WM team Upgrades

Comments

@amaltaro
Copy link
Contributor

amaltaro commented Jul 6, 2022

Impact of the new feature
WMCore (and any other projects depending on the cmsdist spec file)

Is your feature request related to a problem? Please describe.
No problem, provided that we don't get too much behind with these third-party libraries updates.
Upgrading the SQLAlchemy library might be challenging, past experience shows that it usually requires WMCore baseline code changes for new features and or feature changes, so I decided to separate this upgrade in its own ticket.

Describe the solution you'd like
We should upgrade SQLAlchemy to one of the latest stable releases, to be checked in Pypi.
This is the spec file that we need to update:
https://github.com/cms-sw/cmsdist/blob/comp_gcc630/py3-sqlalchemy.spec
and also provide a PR updating our requirements.txt file.

Describe alternatives you've considered
We better not accumulate to many changes between upgrades, so the sooner the better.

Additional context
Similar to what has been requested in #11080

@amaltaro
Copy link
Contributor Author

For further reference, the strategy=threadlocal had been deprecated in 1.3.x SQLAlchemy and it was removed in 1.4.x, which caused jenkins to fail massively with an error like [1], once upgraded to 1.4.40.

Further information is available here:
https://docs.sqlalchemy.org/en/14/changelog/changelog_14.html#change-b922b1e71ec804bcec35652cbfdcf120
and
https://docs.sqlalchemy.org/en/13/core/connections.html#using-the-threadlocal-execution-strategy

[1]

Traceback (most recent call last):
  File "/build/cmsbld/jenkins/workspace/DMWM-WMCorePy3-PR-unittests/SLICE/0/label/cms-dmwm-cc7/deploy/2.1.1.pre3-comp2/sw/slc7_amd64_gcc630/external/python3/3.8.2-comp/lib/python3.8/unittest/case.py", line 60, in testPartExecutor
    yield
  File "/build/cmsbld/jenkins/workspace/DMWM-WMCorePy3-PR-unittests/SLICE/0/label/cms-dmwm-cc7/deploy/2.1.1.pre3-comp2/sw/slc7_amd64_gcc630/external/python3/3.8.2-comp/lib/python3.8/unittest/case.py", line 672, in run
    self._callSetUp()
  File "/build/cmsbld/jenkins/workspace/DMWM-WMCorePy3-PR-unittests/SLICE/0/label/cms-dmwm-cc7/deploy/2.1.1.pre3-comp2/sw/slc7_amd64_gcc630/external/python3/3.8.2-comp/lib/python3.8/unittest/case.py", line 630, in _callSetUp
    self.setUp()
  File "/build/cmsbld/jenkins/workspace/DMWM-WMCorePy3-PR-unittests/SLICE/0/label/cms-dmwm-cc7/code/test/python/WMCore_t/WMBS_t/JobSplitting_t/WMBSMergeBySize_t.py", line 32, in setUp
    self.testInit.setDatabaseConnection()
  File "/build/cmsbld/jenkins/workspace/DMWM-WMCorePy3-PR-unittests/SLICE/0/label/cms-dmwm-cc7/code/src/python/WMQuality/TestInit.py", line 149, in setDatabaseConnection
    self.init.setDatabaseConnection(config.CoreDatabase.connectUrl,
  File "/build/cmsbld/jenkins/workspace/DMWM-WMCorePy3-PR-unittests/SLICE/0/label/cms-dmwm-cc7/code/src/python/WMCore/WMInit.py", line 131, in setDatabaseConnection
    myThread.dbFactory = DBFactory(logging, dbConfig, options)
  File "/build/cmsbld/jenkins/workspace/DMWM-WMCorePy3-PR-unittests/SLICE/0/label/cms-dmwm-cc7/code/src/python/WMCore/Database/DBFactory.py", line 84, in __init__
    create_engine(self.dburl,
  File "<string>", line 2, in create_engine
  File "/build/cmsbld/jenkins/workspace/DMWM-WMCorePy3-PR-unittests/SLICE/0/label/cms-dmwm-cc7/deploy/2.1.1.pre3-comp2/sw/slc7_amd64_gcc630/external/py3-sqlalchemy/1.4.40/lib/python3.8/site-packages/sqlalchemy/util/deprecations.py", line 309, in warned
    return fn(*args, **kwargs)
  File "/build/cmsbld/jenkins/workspace/DMWM-WMCorePy3-PR-unittests/SLICE/0/label/cms-dmwm-cc7/deploy/2.1.1.pre3-comp2/sw/slc7_amd64_gcc630/external/py3-sqlalchemy/1.4.40/lib/python3.8/site-packages/sqlalchemy/engine/create.py", line 513, in create_engine
    raise exc.ArgumentError("unknown strategy: %r" % strat)
sqlalchemy.exc.ArgumentError: unknown strategy: 'threadlocal'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement Medium Priority Technical Debt Used to track issues that address technical needs internal to WM team Upgrades
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants