Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Snowflake #267

Merged
merged 2 commits into from
Dec 11, 2020
Merged

Support Snowflake #267

merged 2 commits into from
Dec 11, 2020

Conversation

kanata2
Copy link
Contributor

@kanata2 kanata2 commented Dec 10, 2020

This is an initial implementation for Snowflake.
(If snowflake support is not needed for tbls, please close this PR 🙏 )

Note

I'm not sure what to do with the testing environment.
So I don't add test cases to snowflake driver yet.

Demo

The following is DDLs and configurations.

create schema tbls_sample;
use schema tbls_sample;
create or replace table tbls_sample (
  id int primary key
  , ext_id int
  , raw variant
  , created_at timestamp_ltz
  , updated_at timestamp_ltz
  , unique (ext_id)
);
dsn: snowflake:/<user>:<password>@<account_id>.ap-northeast-1.aws/SANDBOX/TBLS_SAMPLE
docPath: doc/schema
TBLS_SAMPLE.md # TBLS_SAMPLE

Description

Table Definition
create or replace TABLE TBLS_SAMPLE (
	ID NUMBER(38,0) NOT NULL,
	EXT_ID NUMBER(38,0),
	RAW VARIANT,
	CREATED_AT TIMESTAMP_LTZ(9),
	UPDATED_AT TIMESTAMP_LTZ(9),
	unique (EXT_ID),
	primary key (ID)
);

Columns

Name Type Default Nullable Children Parents Comment
ID NUMBER false
EXT_ID NUMBER true
RAW VARIANT true
CREATED_AT TIMESTAMP_LTZ true
UPDATED_AT TIMESTAMP_LTZ true

Relations

er


Generated by tbls

</details>

This is an initial implementation.
So we do not implement about constraints information yet.
@codecov
Copy link

codecov bot commented Dec 10, 2020

Codecov Report

Merging #267 (9ccbb93) into master (670dc8a) will decrease coverage by 1.35%.
The diff coverage is 0.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #267      +/-   ##
==========================================
- Coverage   68.72%   67.37%   -1.36%     
==========================================
  Files          23       23              
  Lines        3831     3859      +28     
==========================================
- Hits         2633     2600      -33     
- Misses        940     1005      +65     
+ Partials      258      254       -4     
Impacted Files Coverage Δ
datasource/datasource.go 21.96% <0.00%> (-12.96%) ⬇️
schema/schema.go 29.10% <0.00%> (-4.04%) ⬇️
config/config.go 51.35% <0.00%> (-0.56%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 670dc8a...9ccbb93. Read the comment docs.

@k1LoW
Copy link
Owner

k1LoW commented Dec 10, 2020

@kanata2 GREAT WORK!!!!! Thank you!!!!!

I'm not sure what to do with the testing environment.

I'm not able to have a persistent test environment for snowflake as well as you.

If I put snowflake support in tbls, is it ok to make it "experimental support"?

Also, is it possible to commit the following files?

  1. a sample DDL similar to other data sources ( in testdata/ )
  2. a sample document generated by tbls doc for the database created in 1 ( in sample/ )

@k1LoW k1LoW added the enhancement New feature or request label Dec 10, 2020
@kanata2
Copy link
Contributor Author

kanata2 commented Dec 10, 2020

Thank you!

If I put snowflake support in tbls, is it ok to make it "experimental support"?

I think it's good. Thank you for your consideration 🙏 🙏

Also, is it possible to commit the following files?

Okay, I'll add these files later.

@k1LoW
Copy link
Owner

k1LoW commented Dec 10, 2020

Hi @kanata2

  1. a sample DDL similar to other data sources ( in testdata/ )

If you have access to SNOWFLAKE_SAMPLE_DATA, use it. No new DDL is needed.

@kanata2
Copy link
Contributor Author

kanata2 commented Dec 10, 2020

Ah, it's good idea!
I'll use SNOWFLAKE_SAMPLE_DATA.TPCH_SF1.

@kanata2
Copy link
Contributor Author

kanata2 commented Dec 10, 2020

I'm sorry.
In shared database, it seems that the GET_DDL function does not allow, so I'll add self-created DDL based on SNOWFLAKE_SAMPLE_DATA and its result this time.
If there are many requests to run tbls against a shared DB in the future, I'll switch to an implementation that doesn't use GET_DDL.(What do you think?)

@k1LoW
Copy link
Owner

k1LoW commented Dec 10, 2020

@kanata2
Looks good. No problem. 👍

@kanata2
Copy link
Contributor Author

kanata2 commented Dec 10, 2020

Just added DDL and output files in 9ccbb93.

@k1LoW
Copy link
Owner

k1LoW commented Dec 11, 2020

@kanata2 Thank you for your great commit!!

@k1LoW k1LoW merged commit 5ee6d3c into k1LoW:master Dec 11, 2020
@kanata2 kanata2 deleted the snowflake branch December 11, 2020 13:32
@k1LoW k1LoW mentioned this pull request Mar 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants