Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doris illegal label fix #421

Closed
wants to merge 1 commit into from
Closed

Conversation

monologuist
Copy link

@monologuist monologuist commented Jul 10, 2024

Proposed changes

Issue Number: close #420

Problem Summary:

Backgroud(why):
When flink cdc uses doris-flink-connector for data synchronization, it generates random labels to complete streamload. Currently, the generated labels contain illegal "-" hyphens, which makes the labels unable to be cleaned.

Describe the overview of changes.
Replace the "-" in label with "_" to ensure the generation of a legal random label.

Checklist(Required)

  1. Does it affect the original behavior: (Yes)
  2. Has unit tests been added: (No Need)
  3. Has document been added or modified: (No Need)
  4. Does it need to update dependencies: (No)
  5. Are there any changes that cannot be rolled back: (No)

@monologuist
Copy link
Author

@JNSimba @CalvinKirs Hi, can you review my code? I think this is a problem and needs to be fixed

@DongLiang-0
Copy link
Contributor

Thanks for trying to resolve this issue.

I looked at doris's use of this section and found that it is CLEAN LABEL [label] FROM db; This SQL has some syntax problems. For example, pure numbers such as 123 are not supported . This needs to be changed to doris' code.

Generally speaking, if you need to clean the label under db, you can use CLEAN LABEL FROM db; clear all labels

@monologuist
Copy link
Author

Thanks for trying to resolve this issue.

I looked at doris's use of this section and found that it is CLEAN LABEL [label] FROM db; This SQL has some syntax problems. For example, pure numbers such as 123 are not supported . This needs to be changed to doris' code.

Generally speaking, if you need to clean the label under db, you can use CLEAN LABEL FROM db; clear all labels

@DongLiang-0 Thank you for your reply! Do you mean that the SQL CLEAN LABEL label1 FROM db1;restriction on label naming, should be solved in Doris project? Am I right?

I will raise this issue in Doris project, but if doris rejects this proposal, I think we need to consider solving this issue in doris-flink-connector. Because when the flink cdc job failover, a large number of useless labels that cannot be deleted are eventually generated. (When I don't want to directly use CLEAN LABEL FROM db1; to clear all labels)🙂

@DongLiang-0
Copy link
Contributor

@DongLiang-0 Thank you for your reply! Do you mean that the SQL CLEAN LABEL label1 FROM db1;restriction on label naming, should be solved in Doris project? Am I right?

I will raise this issue in Doris project, but if doris rejects this proposal, I think we need to consider solving this issue in doris-flink-connector. Because when the flink cdc job failover, a large number of useless labels that cannot be deleted are eventually generated. (When I don't want to directly use CLEAN LABEL FROM db1; to clear all labels)🙂

@monologuist Yes, this problem needs to be solved in doris to lift the grammatical restrictions on label.

@JNSimba
Copy link
Member

JNSimba commented Jul 24, 2024

I agree with @DongLiang-0 suggestion. It would be better to modify it on Doris side, because the client is not only Flink. Close this first, and you can open it at any time if you have any questions.

@JNSimba JNSimba closed this Jul 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] Wrong label random generation method leads to illegal doris label
3 participants