DRAFT - Migrate from psycopg2 to psycopg3 #987
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Looks like there are some file conflicts? Also, I thought we'd fixed it so the tests would get run against PRs to major-release? If we can get this to run the tests, and the tests pass, we can merge. Though I will do so warily regardless since psycopg has historically caused a huge amount of installation errors. Actually it'd be great if we could test this against an M1 mac, @austinweisgrau do you happen to have one you could test manually? |
shaunagm
requested changes
Feb 6, 2024
|
I believe the major release branch needs to be brought up to date and that should simplify the diffs considerably |
psycopg3 is mostly built as a drop-in upgrade from psycopg2. Differences are detailed here: https://www.psycopg.org/psycopg3/docs/basic/from_pg2.html A few highlights: * Cancelling running queries One major DX improvement is that Ctrl-C can now cancel a running query. With psycopg2, the python process can be killed but a running query usually could not be killed (without knowing the PID and running `pg_cancel_backend(pid)` from a separate connection), to the immediate regret of all we who sometimes accidentally execute an unfortunately large query. See https://www.psycopg.org/psycopg3/docs/advanced/async.html#interrupting-async-operations * Breaking changes to parameterization A few parameterization patterns that worked with psycopg2 no longer work with psycopg3. They are detailed on the migration guide linked above, but these in particular should be highlighted: ** You can no longer use `IN %s` with a tuple This would work with psycopg2 but will not work with psycopg3: `conn.execute("SELECT * FROM foo WHERE id IN %s", parameters=[(10,20,30)])` Instead, this format is advised: `conn.execute("SELECT * FROM foo WHERE id = ANY(%s)", parameters=[[10,20,30]])` ** You can no longer use `IS %s` This would work with psycopg2 but will not work with psycopg3: `conn.execute("SELECT * FROM foo WHERE field IS %s", [None])` Instead, `IS NOT DISTINCT FROM %s` should be used `conn.execute("SELECT * FROM foo WHERE field IS NOT DISTINCT FROM %s", [None])` * Breaking changes to psycopg2 `copy` methods `copy` methods are refactored and consolidated in psycopg3. See the migration guide for details.
austinweisgrau
force-pushed
the
psycopg3
branch
from
7f96322 to
ecdb963
Compare
austinweisgrau
changed the title
|
Ahh it needs more configuration anyways to work propertly, making this a DRAFT for now |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
psycopg3 is mostly built as a drop-in upgrade from psycopg2.
Differences are detailed here: https://www.psycopg.org/psycopg3/docs/basic/from_pg2.html
A few highlights:
One major DX improvement is that Ctrl-C can now cancel a running
query. With psycopg2, the python process can be killed but a running
query usually could not be killed (without knowing the PID and running
pg_cancel_backend(pid)from a separate connection), to the immediateregret of all we who sometimes accidentally execute an unfortunately
large query.
See https://www.psycopg.org/psycopg3/docs/advanced/async.html#interrupting-async-operations
A few parameterization patterns that worked with psycopg2 no longer
work with psycopg3. They are detailed on the migration guide linked
above, but these in particular should be highlighted:
** You can no longer use
IN %swith a tupleThis would work with psycopg2 but will not work with psycopg3:
conn.execute("SELECT * FROM foo WHERE id IN %s", parameters=[(10,20,30)])Instead, this format is advised:
conn.execute("SELECT * FROM foo WHERE id = ANY(%s)", parameters=[[10,20,30]])** You can no longer use
IS %sThis would work with psycopg2 but will not work with psycopg3:
conn.execute("SELECT * FROM foo WHERE field IS %s", [None])Instead,
IS NOT DISTINCT FROM %sshould be usedconn.execute("SELECT * FROM foo WHERE field IS NOT DISTINCT FROM %s", [None])copymethodscopymethods are refactored and consolidated in psycopg3. See themigration guide for details.