Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: flaky parquet predicate suits #307

Merged
merged 2 commits into from
Oct 13, 2022
Merged

fix: flaky parquet predicate suits #307

merged 2 commits into from
Oct 13, 2022

Conversation

waynexia
Copy link
Member

Signed-off-by: Ruihang Xia [email protected]

I notice that some cases in table/predicate will panic, like https://github.com/GreptimeTeam/greptimedb/actions/runs/3232216860/jobs/5292594794

I try to fix it by adding a flush call on closing. But I'm not sure if this is fixed 🫣 I've run it 10+ times in my local env and no more failure occur. Please let me know if it doesn't work.

@v0y4g3r
Copy link
Contributor

v0y4g3r commented Oct 12, 2022

Not sure if it really fixes the flushing issue. afaik it's related to parquet2's end_file method. I opened a PR to fix this.

@waynexia
Copy link
Member Author

Looks like the same fix lol.

@v0y4g3r
Copy link
Contributor

v0y4g3r commented Oct 12, 2022

Also fix the related workaround inside ParquetWriter::write_rows?

@codecov
Copy link

codecov bot commented Oct 12, 2022

Codecov Report

Merging #307 (f5943db) into develop (0c829a9) will increase coverage by 0.02%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##           develop     #307      +/-   ##
===========================================
+ Coverage    84.25%   84.28%   +0.02%     
===========================================
  Files          319      319              
  Lines        28954    28975      +21     
===========================================
+ Hits         24396    24422      +26     
+ Misses        4558     4553       -5     
Flag Coverage Δ
rust 84.28% <100.00%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/storage/src/sst/parquet.rs 86.20% <100.00%> (+0.09%) ⬆️
src/table/src/predicate.rs 92.80% <100.00%> (+0.11%) ⬆️
src/catalog/src/manager.rs 58.33% <0.00%> (-1.67%) ⬇️
src/datatypes/src/vectors/primitive.rs 95.18% <0.00%> (+0.03%) ⬆️
src/datatypes/src/vectors/operations/replicate.rs 98.52% <0.00%> (+0.45%) ⬆️
src/servers/src/opentsdb.rs 97.61% <0.00%> (+4.76%) ⬆️
src/servers/src/error.rs 29.50% <0.00%> (+11.47%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@waynexia
Copy link
Member Author

Also fix the related workaround inside ParquetWriter::write_rows?

done 😄

Copy link
Contributor

@v0y4g3r v0y4g3r left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@evenyag evenyag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@evenyag evenyag merged commit b61d598 into develop Oct 13, 2022
@evenyag evenyag deleted the fix-parquet-test branch October 13, 2022 06:00
paomian pushed a commit to paomian/greptimedb that referenced this pull request Oct 19, 2023
* fix: flaky parquet predicate suits

Signed-off-by: Ruihang Xia <[email protected]>

* fix: change ParquetWriter::write_rows as well

Signed-off-by: Ruihang Xia <[email protected]>

Signed-off-by: Ruihang Xia <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants