Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch mysql db store to postgres #1734

Closed
cobolbaby opened this issue Jul 16, 2020 · 9 comments · Fixed by #1742
Closed

Switch mysql db store to postgres #1734

cobolbaby opened this issue Jul 16, 2020 · 9 comments · Fixed by #1742
Assignees
Labels
accepted An Issue that is confirmed as a bug by the DataHub Maintainers. feature-request Request for a new feature to be added

Comments

@cobolbaby
Copy link
Contributor

cobolbaby commented Jul 16, 2020

Considering the technology stack used, the company made it clear that pg should be used as a unified storage.
So I need to switch mysql to pg. But when I start the gms service, it throws execption No org.postgresql.Driver Class.
Could the relevant dependencies be packaged in the compilation process to facilitate switching jdbc driver.

@cobolbaby cobolbaby added the feature-request Request for a new feature to be added label Jul 16, 2020
@keremsahin1
Copy link
Contributor

This should ideally be pretty straightforward because we provide DAOs for Ebean. Make sure you update Ebean configs to access to your PostgreSQL server. You also need to replace below line for the JDBC connector dependency for PostgreSQL.

https://github.com/linkedin/datahub/blob/master/gms/war/build.gradle#L10

@mars-lan
Copy link
Contributor

@cobolbaby did you have a chance to try @keremsahin1's suggestion? I also assume our Ebean-based DAO can talk to Postgres without much trouble.

@cobolbaby
Copy link
Contributor Author

cobolbaby commented Jul 22, 2020

@clojurians-org has switched the postgresql db in the forked project.

Commit: clojurians-org/simple-datahub@6e678bb

We need to modify the code:

  • related to externalDependency.mysqlConnector
  • EBEAN_DATASOURCE_DRIVER
  • EBEAN_DATASOURCE_URL

Is it possible to do some compatible processing?

@mars-lan
Copy link
Contributor

mars-lan commented Jul 22, 2020

Cool. Could you confirm that the doc here is sufficient?

https://github.com/linkedin/datahub/blob/master/gms/README.md#using-a-different-db-backend

@cobolbaby
Copy link
Contributor Author

After searching externalDependency.mysqlConnector globally, I found three places that need to be modified:

  • datahub-frontend/play.gradle
  • gms/war/build.gradle
  • build.gradle

In addition, the field type in init.sql needs to be adjusted

@cobolbaby
Copy link
Contributor Author

Is it possible to package the pg driver by default?

@mars-lan
Copy link
Contributor

One option is to make it a gradle build option so you can choose to package mysql or pg (or any others DB systems in the future).

@cobolbaby
Copy link
Contributor Author

cobolbaby commented Jul 22, 2020

If adopting this method, how to release docker images?

@mars-lan
Copy link
Contributor

Good point. Let me think about it a bit more and create a proper PR for this.

@mars-lan mars-lan self-assigned this Jul 22, 2020
@mars-lan mars-lan added the accepted An Issue that is confirmed as a bug by the DataHub Maintainers. label Jul 22, 2020
chriscollins3456 pushed a commit to chriscollins3456/datahub that referenced this issue Sep 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accepted An Issue that is confirmed as a bug by the DataHub Maintainers. feature-request Request for a new feature to be added
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants