Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

opendal: Bump to v0.0.5 to address performance regression and much UE enhancement #4203

Merged
merged 8 commits into from
Feb 24, 2022

Conversation

Xuanwo
Copy link
Member

@Xuanwo Xuanwo commented Feb 18, 2022

Signed-off-by: Xuanwo [email protected]

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

This PR intends to adapt the opendal API changes, Fix #4197

More details could be found in apache/opendal#35

Changelog

  • Performance Improvement

Test Plan

Unit Tests

Stateless Tests

@vercel
Copy link

vercel bot commented Feb 18, 2022

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/databend/databend/369TEd6M964EBSEBNJixXjJ6Hnun
✅ Preview: https://databend-git-fork-xuanwo-opendal-bench-databend.vercel.app

[Deployment for 78a2cc7 canceled]

@mergify
Copy link
Contributor

mergify bot commented Feb 18, 2022

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

@Xuanwo Xuanwo marked this pull request as ready for review February 23, 2022 11:08
@Xuanwo Xuanwo requested a review from BohuTANG as a code owner February 23, 2022 11:08
@Xuanwo
Copy link
Member Author

Xuanwo commented Feb 23, 2022

Our API looks much better! ping @sundy-li @dantengsky for benchmark test.

@Xuanwo Xuanwo changed the title opendal: Adapt to latest opendal API opendal: Bump to v0.0.5 to address performance regression Feb 23, 2022
Signed-off-by: Xuanwo <[email protected]>
@sundy-li
Copy link
Member

Performance not expected, make this draft temporary.

@BohuTANG
Copy link
Member

Performance not expected, make this draft temporary.

How about the performance?
The OpenDAL 0.5 looks not only related to the performance (I think the PR title misleading here), also improve error catch and some API design.
cc @Xuanwo

@Xuanwo
Copy link
Member Author

Xuanwo commented Feb 23, 2022

Merged with main to make sure we bench on the same base.

@Xuanwo
Copy link
Member Author

Xuanwo commented Feb 23, 2022

Current main branch:

commit 30c9c863ab43d8bdeba7443b1ca2e7bad11e93c9 (HEAD -> main, origin/main, origin/HEAD)
Merge: af25873b0 75534d796
Author: mergify[bot] <37929162+mergify[bot][@users.noreply.github.com](http://users.noreply.github.com/)>
Date:   Wed Feb 23 11:08:57 2022 +0000

    Merge pull request #4223 from lichuang/lichuang_refactor

    Refactor: add retry join cluster process, refactor --join arg format
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `cat /tmp/bench_0.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 4.1 ± 0.3 | 3.7 | 4.6 | 1.00 |
| `cat /tmp/bench_1.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 485.3 ± 5.4 | 480.4 | 498.3 | 117.51 ± 9.65 |
| `cat /tmp/bench_2.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 509.3 ± 6.2 | 502.6 | 521.6 | 123.34 ± 10.14 |
| `cat /tmp/bench_3.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 484.2 ± 6.7 | 472.4 | 495.2 | 117.25 ± 9.67 |
| `cat /tmp/bench_4.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 699.4 ± 7.9 | 690.2 | 716.8 | 169.38 ± 13.90 |
| `cat /tmp/bench_5.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 617.3 ± 7.4 | 605.8 | 628.0 | 149.48 ± 12.28 |
| `cat /tmp/bench_6.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 584.9 ± 7.0 | 576.0 | 599.7 | 141.64 ± 11.64 |
| `cat /tmp/bench_7.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 856.0 ± 9.3 | 843.9 | 872.9 | 207.30 ± 17.00 |
| `cat /tmp/bench_8.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 906.4 ± 4.5 | 900.3 | 916.1 | 219.51 ± 17.88 |
| `cat /tmp/bench_9.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 796.4 ± 11.7 | 782.2 | 821.5 | 192.87 ± 15.94 |
| `cat /tmp/bench_10.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 770.4 ± 9.9 | 756.5 | 783.3 | 186.57 ± 15.36 |
| `cat /tmp/bench_11.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 966.7 ± 14.7 | 952.1 | 996.6 | 234.12 ± 19.37 |
| `cat /tmp/bench_12.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 1148.9 ± 13.6 | 1124.2 | 1169.0 | 278.24 ± 22.86 |
| `cat /tmp/bench_13.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 1049.9 ± 7.9 | 1039.1 | 1069.4 | 254.25 ± 20.76 |

This PR:

commit 9447df5737bebfcbd96f48e8be095e0764518c36 (HEAD -> opendal-bench, xuanwo/opendal-bench)
Merge: c51b6a993 30c9c863a
Author: Xuanwo <[email protected]>
Date:   Wed Feb 23 21:37:39 2022 +0800

    Merge remote-tracking branch 'origin/main' into opendal-bench
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `cat /tmp/bench_0.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 4.2 ± 0.4 | 3.7 | 5.1 | 1.00 |
| `cat /tmp/bench_1.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 251.3 ± 6.6 | 238.9 | 257.7 | 59.60 ± 5.84 |
| `cat /tmp/bench_2.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 269.9 ± 3.9 | 264.0 | 277.0 | 64.02 ± 6.11 |
| `cat /tmp/bench_3.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 249.2 ± 4.3 | 240.4 | 254.4 | 59.10 ± 5.67 |
| `cat /tmp/bench_4.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 404.9 ± 10.2 | 394.4 | 424.4 | 96.02 ± 9.38 |
| `cat /tmp/bench_5.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 365.3 ± 7.7 | 356.2 | 380.8 | 86.63 ± 8.38 |
| `cat /tmp/bench_6.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 315.3 ± 1.8 | 312.6 | 317.8 | 74.77 ± 7.07 |
| `cat /tmp/bench_7.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 510.4 ± 9.1 | 497.6 | 532.6 | 121.04 ± 11.62 |
| `cat /tmp/bench_8.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 507.7 ± 4.7 | 503.3 | 519.1 | 120.42 ± 11.42 |
| `cat /tmp/bench_9.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 492.3 ± 6.3 | 485.3 | 506.2 | 116.76 ± 11.12 |
| `cat /tmp/bench_10.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 467.6 ± 5.9 | 456.7 | 479.0 | 110.90 ± 10.56 |
| `cat /tmp/bench_11.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 566.7 ± 8.7 | 554.7 | 583.8 | 134.41 ± 12.85 |
| `cat /tmp/bench_12.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 680.2 ± 5.4 | 671.3 | 687.3 | 161.33 ± 15.28 |
| `cat /tmp/bench_13.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 600.5 ± 4.3 | 595.7 | 607.7 | 142.42 ± 13.48 |

Old query: DatabendQuery v-0.1.0-d2256c1-simd(1.60.0-nightly-2022-02-16T01:58:34.837999309+00:00)

| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `cat /tmp/bench_0.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 4.4 ± 0.4 | 3.9 | 5.1 | 1.00 |
| `cat /tmp/bench_1.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 249.5 ± 3.2 | 245.6 | 254.7 | 56.53 ± 5.34 |
| `cat /tmp/bench_2.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 268.7 ± 3.9 | 263.6 | 274.4 | 60.89 ± 5.77 |
| `cat /tmp/bench_3.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 242.2 ± 5.4 | 232.3 | 252.9 | 54.88 ± 5.28 |
| `cat /tmp/bench_4.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 386.1 ± 8.6 | 375.1 | 401.1 | 87.48 ± 8.41 |
| `cat /tmp/bench_5.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 352.7 ± 5.3 | 344.3 | 361.3 | 79.91 ± 7.57 |
| `cat /tmp/bench_6.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 303.7 ± 6.5 | 295.7 | 317.9 | 68.82 ± 6.61 |
| `cat /tmp/bench_7.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 464.2 ± 8.9 | 448.4 | 478.9 | 105.19 ± 10.05 |
| `cat /tmp/bench_8.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 474.2 ± 5.9 | 463.8 | 484.1 | 107.44 ± 10.14 |
| `cat /tmp/bench_9.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 493.5 ± 9.1 | 477.9 | 506.5 | 111.81 ± 10.66 |
| `cat /tmp/bench_10.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 465.3 ± 9.9 | 447.1 | 476.0 | 105.43 ± 10.12 |
| `cat /tmp/bench_11.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 528.7 ± 12.7 | 513.0 | 550.0 | 119.81 ± 11.57 |
| `cat /tmp/bench_12.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 655.5 ± 9.4 | 641.9 | 667.4 | 148.52 ± 14.06 |
| `cat /tmp/bench_13.sql \| mysql --host 127.0.0.1 -P 3307 -uroot -s ` | 590.8 ± 15.7 | 563.7 | 609.6 | 133.87 ± 13.03 |

This PR will fix the performance regression introduced by opendal v0.0.3, and we will fig out how to improve from d2256c1.

ping @sundy-li to take a look again.

@Xuanwo Xuanwo marked this pull request as ready for review February 23, 2022 14:12
Signed-off-by: Xuanwo <[email protected]>
@BohuTANG
Copy link
Member

Oops, has conflicts that must be resolved.

@Xuanwo
Copy link
Member Author

Xuanwo commented Feb 23, 2022

Oops, has conflicts that must be resolved.

Fixed.

@Xuanwo Xuanwo changed the title opendal: Bump to v0.0.5 to address performance regression opendal: Bump to v0.0.5 to address performance regression and much UE enhancement Feb 23, 2022
@Xuanwo
Copy link
Member Author

Xuanwo commented Feb 23, 2022

The OpenDAL 0.5 looks not only related to the performance (I think the PR title misleading here), also improve error catch and some API design.

PR title updated 😆

@BohuTANG
Copy link
Member

/lgtm

@mergify mergify bot merged commit a62eb7f into databendlabs:main Feb 24, 2022
@Xuanwo Xuanwo deleted the opendal-bench branch February 24, 2022 01:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

performance drop 3 times after bump up opendal
4 participants