Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tracking issue for flaky tests #2246

Closed
Rustin170506 opened this issue Jul 8, 2021 · 2 comments
Closed

Tracking issue for flaky tests #2246

Rustin170506 opened this issue Jul 8, 2021 · 2 comments
Labels
area/dm Issues or PRs related to DM. area/ticdc Issues or PRs related to TiCDC. component/test Unit tests and integration tests component. help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. priority/P1 The issue has P1 priority. severity/minor type/bug The issue is confirmed as a bug.

Comments

@Rustin170506
Copy link
Member

Rustin170506 commented Jul 8, 2021 •
edited by okJiang
Loading

This is a tracking issue for unstable tests, and each unstable test is linked as a subtask.

TiCDC

DM

If you encounter the unstable test repeatedly, you can add +n in the end of following test. For example,

We will give priority to frequent unstable tests.

Engine
@Rustin170506 Rustin170506 added the type/bug The issue is confirmed as a bug. label Jul 8, 2021
@Rustin170506 Rustin170506 mentioned this issue Jul 8, 2021
Closed
@Rustin170506 Rustin170506 added priority/P1 The issue has P1 priority. severity/major component/test Unit tests and integration tests component. help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. labels Jul 8, 2021
@Rustin170506
Copy link
Member Author

Rustin170506 commented Jul 8, 2021 •
edited
Loading

Basic guidelines for fixing these tests.

  1. discover unstable tests.
  2. Create a corresponding issue to track the unstable tests.
    3. Submit a PR to temporarily skip the unstable test. (The expected time to fix must be set in the PR)
  3. start fixing the unstable tests. (assign the issue to someone)
    5. Bring the test back online.

First, our integration tests need to support skipping some unstable tests, and I will adapt the sh script to support this mechanism.

@overvenus overvenus pinned this issue Jul 13, 2021
@asddongmen asddongmen unpinned this issue Aug 5, 2021
@amyangfei amyangfei pinned this issue Aug 6, 2021
@Rustin170506 Rustin170506 changed the title Tracking issue for unstable tests Tracking issue for flaky tests Sep 8, 2021
@Rustin170506 Rustin170506 mentioned this issue Sep 13, 2021
Closed
@Rustin170506 Rustin170506 added area/dm Issues or PRs related to DM. area/ticdc Issues or PRs related to TiCDC. labels Nov 5, 2021
@Rustin170506 Rustin170506 mentioned this issue Aug 2, 2022
Merged
ti-chi-bot pushed a commit that referenced this issue Aug 2, 2022
@Rustin170506
tests(ticdc): increase waiting time for Canal JSON case (#6568)
67f7e3f 
ref #2246, ref #6493
@Rustin170506 Rustin170506 mentioned this issue Oct 17, 2022
Merged
ti-chi-bot pushed a commit that referenced this issue Oct 18, 2022
@Rustin170506
github(ticdc): fix broken test (#7370)
e3acc1a 
ref #2246, ref pingcap/tidb#38494
@GMHDBJD GMHDBJD mentioned this issue Nov 22, 2022
Merged
@D3Hunter D3Hunter mentioned this issue Dec 5, 2022
Merged
ti-chi-bot pushed a commit that referenced this issue Dec 5, 2022
@D3Hunter
test(cdc): fix canal integration (#7802)
e68f91a 
ref #2246
@amyangfei amyangfei mentioned this issue Dec 5, 2022
Merged
ti-chi-bot pushed a commit that referenced this issue Dec 7, 2022
@amyangfei
test(ticdc): add centos-base.repo file directly in canal test (#7812)
c546945 
ref #2246
@jebter jebter unpinned this issue Feb 2, 2023
@Rustin170506 Rustin170506 mentioned this issue May 24, 2023
Merged
ti-chi-bot bot pushed a commit that referenced this issue May 30, 2023
@Rustin170506
tests(ticdc): add more test for lossy_ddl (#9039)
02e2bd3 
ref #2246
@Rustin170506 Rustin170506 mentioned this issue Jun 12, 2023
Merged
ti-chi-bot bot pushed a commit that referenced this issue Jun 12, 2023
@Rustin170506
tests(ticdc): fix the flaky test (#9191)
a912663 
ref #2246
@asddongmen
Copy link
Contributor

Great works! I will close it since it is stale.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/dm Issues or PRs related to DM. area/ticdc Issues or PRs related to TiCDC. component/test Unit tests and integration tests component. help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. priority/P1 The issue has P1 priority. severity/minor type/bug The issue is confirmed as a bug.
Projects
Question and Bug Reports
Status: Done
Milestone
No milestone
Development

No branches or pull requests
3 participants
@VelocityLight @asddongmen @Rustin170506