Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid running SQLAlchemy schema tests with Ingres dialect #50

Merged
merged 4 commits into from
May 14, 2024

Conversation

hab6
Copy link
Contributor

@hab6 hab6 commented May 9, 2024

When running the SQLAlchemy dialect compliance suite, quite a few tests would fail with some form of the following error:

sqlalchemy.exc.ProgrammingError: (pyodbc.ProgrammingError)
('42503', "[42503] [Actian][Actian II ODBC Driver][INGRES]
CREATE TABLE: You may not create an object owned by 'test_schema'. (328737) (SQLExecDirectW)")
[SQL:
CREATE TABLE test_schema.test_table (
    id INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL,
    data VARCHAR(50),
    PRIMARY KEY (id) )

The reason for the error is that the affected tests attempt to create and drop tables owned under schemas different than the current user.

Ingres does not support this kind of operation except in rare instances where the current user has security administrator and operator privilege or who is the DBA of the database. The user must then use the SET SESSION AUTHORIZATION ... command to set the effective user for the current session.

Per discussion in 11366 and additional research in internal ticket II-14148, the setting supports_schemas = False will be added to the IngresDialect class to avoid execution of tests that use alternate schemas.

The file README.testsuite.md is being added to provide information on how to run the dialect compliance suite and also provide information about the issue of running tests that contain alternate schemas.

Testing

Ingres 11.2 (15807) on Windows 10 (client and server are same machine)
Python 3.10.7
mock 5.1.0
pluggy 1.5.0
psycopg2 2.9.9
pyodbc 5.1.0
pypyodbc 1.3.6
pytest 8.1.1
setuptools 69.5.1
SQLAlchemy 2.0.29.dev0
sqlalchemy-ingres 0.0.7.dev1
tox 4.14.2
Result Baseline With
supports_schemas = False
Diff
Total Tests 1531 1531 -
Passed 336 473 +137
Failed 63 173 +110
Errors 798 30 -768
Skipped 334 855 +521
Run Time 17m 25s 42s -16m 43s

@hab6 hab6 requested review from clach04, mianculovici and vsreeniv May 9, 2024 18:59
@hab6
Copy link
Contributor Author

hab6 commented May 10, 2024

I am suspicious about the huge jump in skipped tests when setting supports_schemas = False in the IngresDialect class.

Result Without
supports_schemas = False
With
supports_schemas = False
Diff
Skipped 334 855 +521

I think it will be important to run the suite with and without the setting and review the 521 tests that are affected to see they are indeed skipped due to the alternate schema issue and not some side effect.

Copy link
Member

@clach04 clach04 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

Needs explanation text (that you wrote in the PR ) in the readme.

Minor tweak suggested for quoting.


## SQLAlchemy Dialect Compliance Suite
### Overview and Setup
SQLAlchemy includes a [dialect compliance suite](https://github.com/sqlalchemy/sqlalchemy/tree/main/lib/sqlalchemy/testing/suite) that is usable by third party libraries, in the source tree at lib/sqlalchemy/testing/suite. There's no need for a third party dialect to run through SQLAlchemy's full testing suite, as a large portion of these tests do not have dialect-sensitive functionality. The "dialect compliance suite" should be viewed as the primary target for new dialects.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quote this, i.e. prefix with >

test_schema
test_schema_2 (only used on PostgreSQL and mssql)

Please refer to your vendor documentation for the proper syntax to create these namespaces - the database user must have permission to create and drop tables within these schemas. Its perfectly fine to run the test suite without these namespaces present, it only means that a handful of tests which expect them to be present will fail.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quote these too, makes it more clear whats a quote (and where from) and our notes

[changelog_14.rst](https://github.com/sqlalchemy/sqlalchemy/blob/main/doc/build/changelog/changelog_14.rst)

**Ingres Dialect Behavior**
The Ingres dialect disables the ability to run tests that use alternate schemas with the setting `supports_schemas = False` in class `IngresDialect`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good statement, it needs an explanation adding (steal the text from your PR, that text is perfect).

@@ -387,6 +387,7 @@ class IngresDialect(default.DefaultDialect):
supports_empty_insert = False
supports_empty_insert = False
supports_statement_cache = False # NOTE this is not actually picked up by SA warning code, _generate_cache_attrs() checks dict of subclass, not the entire class
supports_schemas = False
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommend having a comment here, could simple say "see readme.testsuite.md for details"

@hab6
Copy link
Contributor Author

hab6 commented May 13, 2024

@clach04 Thanks for your review. I added text from the PR to the new README and cleaned it up a bit, including your suggestions for making the quoted text distinct that is taken from the SQLAlchemy repository pages.

Please let me know of additional suggestions. Thanks.

@clach04
Copy link
Member

clach04 commented May 13, 2024

Discussed with @hab6 - he'd like to hold off merge due to >50% tests being skipped. Want to review

@hab6
Copy link
Contributor Author

hab6 commented May 14, 2024

Focusing solely on the tests in dialect compliance suite class ComponentReflectionTest:

Without the flag supports_schemas = False in the IngresDialect class, all reflection tests fail with this error (e.g. involving tables): [INGRES]CREATE TABLE: You may not create an object owned by 'test_schema'. Conversely, with the flag present in the IngresDialect class, the majority of the tests are skipped, yet many tests now fall into the PASS and FAIL categories.

Results of running ComponentReflectionTest class

IngresDialect class
contains flag
supports_schemas = False
PASSED FAILED ERROR SKIPPED TOTAL
No 0 0 732 0 732
Yes 131 104 0 497 732

@hab6 hab6 merged commit 022f3d7 into master May 14, 2024
@hab6 hab6 deleted the hab6-support-schemas-false branch May 14, 2024 21:57
hab6 added a commit that referenced this pull request May 15, 2024
Bump version for PR #50
hab6 added a commit that referenced this pull request May 15, 2024
Bump version for PR #50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants