Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support multiple schema for Sharding-proxy #1046

Closed
13 tasks done
beckhampu opened this issue Jul 25, 2018 · 4 comments
Closed
13 tasks done

Support multiple schema for Sharding-proxy #1046

beckhampu opened this issue Jul 25, 2018 · 4 comments
Assignees

Comments

@beckhampu
Copy link
Contributor

beckhampu commented Jul 25, 2018

Sharding-Proxy just have one default logic database which is sharding_db and it and it can't be configured. Now , We will add multiple logic schema configurations to improve availability!

Here are some details about this implementation:

  1. About configuration, we will pull orchestration and proxyAuthority configurations out and save them in a new configuration file like server.yaml. And the shardingrule configurations then takes the form of one schemal corresponding to one config.yaml file.

  2. About orchestration, we will adjust to add schemalName to the configuration path.

  3. About frontend command processing, when receive the COM_INIT_DB command and sql execution of 'use database' , we save the current schemal in the FrontendHandler.

  4. About SQL execution, when BackendHandler is built, the corresponding RuleRegistry is retrieved from the current schemal for subsequent execution processing. For some SQL statements that do not require of schemal (like 'show databases'), we are currently processing by setting up a default schemal from the configurations.This is not a good solution, we will consider optimizing it later.

task list.

  • load schema sharding rule configuration from 'config-[\w-]+.yaml'.
  • add 'server.yaml' file for loading orchestration, proxyAuthority and some props configurtions.
  • add ProxyContext . It maintains the relationship between schema and RuleRegistry.
  • adjust front handler to save current schema.
  • asjust parsing UseStatement.
  • add ProxyShowDatabasesMergedResult to solve ‘show databases'
  • add BackendNettyClientManager. It maintains the relationship between schema and BackendNettyClient.
  • adjust orchestration for multiple schema.
  • remove default schema, operate with use db just return error packet.
  • use real schema instead of sharding_db in SHOW TABLES results.
  • support db_name on SHOW INDEX Syntax.
  • support db_name on SHOW COLUMNS syntax.
  • support db_name on SHOW TABLE STATUS syntax.
@beckhampu
Copy link
Contributor Author

Here are some coding details.
About config loading:
1、Load the 'config-*.yaml' file to get all the sharding rule configuration of schmal.
2、Load the server.yaml file to get orchestration, proxyAuthority, and props configurations.
3、Add new class ProxyContext. It initialized at startup, saves the all global configurations and maintains the relationship between schema and RuleRegistry.

About front handler:
1、The field schema is added to FrontHandler to save the current schema selected by the user.
2、Check and save schema When received HandshakeResponse41Packet,like 'mysql -u root -proot -h 127.0.0.1 -P3307 sharding_db'.

About execute command:
1、COM_INIT_DB: check and change the schema.
2、COM_FIELD_LIST: get RuleRegistry from the schema and construct the BackendHandler.
3、COM_QUERY: get RuleRegistry from the schema and construct the BackendHandler.
4、COM_STMT_PREPARE: get RuleRegistry from the schema and construct the SqlParsingEngine.
5、COM_STMT_EXECUTE:get RuleRegistry from the schema and construct the BackendHandler.

About sql parsing:
1、Adjust MySQLUseParser to get schema when parsing sql 'use db'.
2、Adjust SQLJudgeEngine to get schema when parsing sql 'use db'.

About JDBCBackendHandler:
1、According to the sql routing result, get and change the schema if returning UseStatement.
2、According to the merge reult, use ProxyShowDatabasesMergedResul instead of ShowDatabasesMergedResult.

About NettyBackendHandler:
1、Each schema stars a BackendNettyClient.
2、Add new class BackendNettyClientManager.It maintains the relationship between schema and BackendNettyClient.
3、According to the sql judge result, get and change the schema if returning UseStatement.
4、Modify method 'mergeDQLorDAL', use ProxyShowDatabasesMergedResul instead of ShowDatabasesMergedResult.

About orchestration:
1、config/proxy/rule: save the sharding rules for all schemas.
2、config/proxy/server: save the global configuration in the server.yaml file
3、state/datasources: add datasource node like 'schema.dbName',such as 'sharding_db.ds0'

@beckhampu
Copy link
Contributor Author

Supported SQL
use db_name
show databases

Unsupported SQL
select database()
select * from db_name.tbl_name

@terrymanu
Copy link
Member

terrymanu commented Sep 15, 2018

I add some new tasks, let's support more DAL

@terrymanu
Copy link
Member

terrymanu commented Sep 15, 2018

I just refactor some places:

  1. Move proxy config from API package to other packages, we need keep API package clear.
  2. config object still little mess, because of reuse and decouple with core and orchestration module, I will continue to do this. (NOT FULLY FINISHED)
  3. Merge SHOW DATABASES's merge engine, make consist with Sharding-JDBC and Sharding-Proxy.

@terrymanu terrymanu changed the title Support multiple schema configuration for Sharding-proxy Support multiple schema for Sharding-proxy Sep 26, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants