unifying vertica connector as jdbc connector #1918
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
AbdulR3hman
requested changes
May 3, 2024
| @@ -17,7 +17,7 @@ | |||
| * limitations under the License. | |||
| * #L% | |||
| */ | |||
| package com.amazonaws.athena.connectors.vertica; | |||
| package com.amazonaws.athena.connectors.vertica.qpt; | |||
There was a problem hiding this comment.
Is this class required any more? Why can't we just do what other SQL connectors are doing and use the QPT class in the JDBC package; specifically this: JdbcQueryPassthrough.java and add the capabilities like MySQLMetadataHandler Line: 139
There was a problem hiding this comment.
Agree. pushed the required changes.
...ertica/src/test/java/com/amazonaws/athena/connectors/vertica/VerticaMetadataHandlerTest.java
Show resolved
Hide resolved
...ertica/src/test/java/com/amazonaws/athena/connectors/vertica/VerticaMetadataHandlerTest.java
Outdated
Show resolved
Hide resolved
...ertica/src/test/java/com/amazonaws/athena/connectors/vertica/VerticaMetadataHandlerTest.java
Outdated
Show resolved
Hide resolved
| @@ -367,11 +328,14 @@ public void getPartitions(BlockWriter blockWriter, GetTableLayoutRequest request | |||
| */ | |||
| @Override | |||
| public GetSplitsResponse doGetSplits(BlockAllocator allocator, GetSplitsRequest request) | |||
| throws SQLException | |||
| { | |||
There was a problem hiding this comment.
it doesn't seem we are handling splits from QPT? Is that on purpose?
There was a problem hiding this comment.
you could use: setupQueryPassthroughSplit from the parent class.
There was a problem hiding this comment.
here there is a somewhat different approach than other JDBC connectors. creating SQL statement from qpt args and exporting results to S3.
AbdulR3hman
approved these changes
May 10, 2024
AbdulR3hman
force-pushed
the
vertica_refactor
branch
from
baa9f40 to
58d60fa
Compare
github-actions bot
pushed a commit
that referenced
this pull request
May 14, 2024
- Revert "build(deps): bump com.google.cloud:google-cloud-bigquery" (#1971) - build(deps): bump com.google.cloud:google-cloud-bigquery from 2.37.0 to 2.40.1 (#1962) - build(deps): bump com.google.cloud:google-cloud-bigquery - build(deps): bump com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1 from 2.47.0 to 3.5.1 (#1961) - build(deps): bump com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1 - build(deps): bump gremlinDriverVersion from 3.6.5 to 3.7.2 (#1967) - build(deps): bump gremlinDriverVersion from 3.6.5 to 3.7.2 - build(deps): bump com.google.cloud:google-cloud-resourcemanager from 1.44.0 to 1.45.0 (#1968) - build(deps): bump com.google.cloud:google-cloud-resourcemanager - build(deps): bump org.elasticsearch.client:elasticsearch-rest-client from 8.13.3 to 8.13.4 (#1970) - build(deps): bump org.elasticsearch.client:elasticsearch-rest-client - build(deps): bump com.amazon.redshift:redshift-jdbc42 from 2.1.0.26 to 2.1.0.27 (#1969) - build(deps): bump com.amazon.redshift:redshift-jdbc42 - build(deps): bump software.amazon.awssdk:bom from 2.25.45 to 2.25.50 (#1965) - build(deps): bump software.amazon.awssdk:bom from 2.25.45 to 2.25.50 - build(deps): bump com.google.cloud:google-cloud-storage from 2.37.0 to 2.38.0 (#1966) - build(deps): bump com.google.cloud:google-cloud-storage - build(deps): bump aws-sdk.version from 1.12.715 to 1.12.720 (#1964) - build(deps): bump aws-sdk.version from 1.12.715 to 1.12.720 - Enable map of complex types support (#1917) - unifying vertica connector as jdbc connector (#1918) - Feature/oracle fips (#1946) - Updated SigV4 signing library in Gremlin connection for Neptune connector (#1698)
github-actions bot
pushed a commit
that referenced
this pull request
May 17, 2024
- Snowflake split issue (#1963) - Com.google.cloud google cloud bigquery version update (#1975) - Fix ValidationException for Timestream connector (#1888) - build(deps): bump com.amazon.redshift:redshift-jdbc42 from 2.1.0.27 to 2.1.0.28 in /athena-redshift (#1974) - build(deps): bump com.amazon.redshift:redshift-jdbc42 - Updated prepare_dev_env.sh script (#1972) - Revert "build(deps): bump com.google.cloud:google-cloud-bigquery" (#1971) - build(deps): bump com.google.cloud:google-cloud-bigquery from 2.37.0 to 2.40.1 (#1962) - build(deps): bump com.google.cloud:google-cloud-bigquery - build(deps): bump com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1 from 2.47.0 to 3.5.1 (#1961) - build(deps): bump com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1 - build(deps): bump gremlinDriverVersion from 3.6.5 to 3.7.2 (#1967) - build(deps): bump gremlinDriverVersion from 3.6.5 to 3.7.2 - build(deps): bump com.google.cloud:google-cloud-resourcemanager from 1.44.0 to 1.45.0 (#1968) - build(deps): bump com.google.cloud:google-cloud-resourcemanager - build(deps): bump org.elasticsearch.client:elasticsearch-rest-client from 8.13.3 to 8.13.4 (#1970) - build(deps): bump org.elasticsearch.client:elasticsearch-rest-client - build(deps): bump com.amazon.redshift:redshift-jdbc42 from 2.1.0.26 to 2.1.0.27 (#1969) - build(deps): bump com.amazon.redshift:redshift-jdbc42 - build(deps): bump software.amazon.awssdk:bom from 2.25.45 to 2.25.50 (#1965) - build(deps): bump software.amazon.awssdk:bom from 2.25.45 to 2.25.50 - build(deps): bump com.google.cloud:google-cloud-storage from 2.37.0 to 2.38.0 (#1966) - build(deps): bump com.google.cloud:google-cloud-storage - build(deps): bump aws-sdk.version from 1.12.715 to 1.12.720 (#1964) - build(deps): bump aws-sdk.version from 1.12.715 to 1.12.720 - Enable map of complex types support (#1917) - unifying vertica connector as jdbc connector (#1918) - Feature/oracle fips (#1946) - Updated SigV4 signing library in Gremlin connection for Neptune connector (#1698)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of changes:
As suggested in qpt review comment, unified the vertical connector code as jdbc connector.
There are some changes in connector documentation as part of this PR. I will share doc separately.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.