-
Notifications
You must be signed in to change notification settings - Fork 142
2021 05 20 bench results
Andrea edited this page May 20, 2021
·
1 revision
Benchmarked on: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz, 32G RAM
Config:
b81da51 Wed, 19 May 2021 23:36:38 +0200 GitHub Merge pull request #145 from LesnyRumcajs/harden-analysis-cleanup
- GRPC_BENCHMARK_DURATION=120s
- GRPC_BENCHMARK_WARMUP=30s
- GRPC_SERVER_CPUS=<1-3>
- GRPC_SERVER_RAM=512m
- GRPC_CLIENT_CONNECTIONS=50
- GRPC_CLIENT_CONCURRENCY=1000
- GRPC_CLIENT_QPS=0
- GRPC_CLIENT_CPUS=9
- GRPC_REQUEST_PAYLOAD=100B
1 CPU Server
-----------------------------------------------------------------------------------------------------------------------------------------
| name | req/s | avg. latency | 90 % in | 95 % in | 99 % in | avg. cpu | avg. memory |
-----------------------------------------------------------------------------------------------------------------------------------------
| rust_tonic_st | 49020 | 20.33 ms | 24.89 ms | 25.97 ms | 28.54 ms | 100.79% | 14.59 MiB |
| rust_tonic_mt | 41840 | 23.83 ms | 10.27 ms | 12.26 ms | 587.59 ms | 101.14% | 15.22 MiB |
| rust_thruster_st | 41615 | 23.96 ms | 29.28 ms | 30.57 ms | 33.38 ms | 100.95% | 12.12 MiB |
| dotnet_grpc | 39031 | 25.52 ms | 32.58 ms | 35.15 ms | 41.80 ms | 99.89% | 94.73 MiB |
| java_quarkus | 36195 | 27.55 ms | 50.99 ms | 54.99 ms | 80.77 ms | 100.36% | 107.48 MiB |
| rust_grpcio | 36132 | 27.61 ms | 30.60 ms | 31.41 ms | 34.33 ms | 102.61% | 25.06 MiB |
| rust_thruster_mt | 34667 | 28.77 ms | 12.22 ms | 14.03 ms | 749.21 ms | 101.23% | 12.79 MiB |
| java_hotspot_grpc_pgc | 34090 | 29.25 ms | 54.16 ms | 65.19 ms | 94.38 ms | 101.51% | 199.68 MiB |
| java_hotspot_grpc_g1gc | 30676 | 32.51 ms | 60.23 ms | 80.72 ms | 97.09 ms | 100.96% | 202.68 MiB |
| cpp_grpc_mt | 30031 | 33.23 ms | 37.64 ms | 38.59 ms | 41.43 ms | 102.53% | 10.03 MiB |
| java_hotspot_grpc_zgc | 29379 | 33.95 ms | 70.55 ms | 88.83 ms | 107.73 ms | 101.45% | 143.01 MiB |
| cpp_grpc_st | 29358 | 33.99 ms | 38.14 ms | 39.24 ms | 41.63 ms | 102.25% | 8.09 MiB |
| java_hotspot_grpc_she | 28249 | 35.31 ms | 64.97 ms | 79.13 ms | 97.97 ms | 101.45% | 405.58 MiB |
| java_openj9_grpc_gencon | 28111 | 35.48 ms | 72.49 ms | 87.83 ms | 99.44 ms | 95.22% | 111.29 MiB |
| java_hotspot_grpc_sgc | 25023 | 39.87 ms | 71.08 ms | 80.18 ms | 101.07 ms | 101.19% | 84.67 MiB |
| crystal_grpc | 21980 | 45.38 ms | 52.88 ms | 55.79 ms | 60.00 ms | 112.54% | 35.16 MiB |
| kotlin_grpc | 17634 | 56.58 ms | 100.40 ms | 107.77 ms | 184.80 ms | 101.15% | 215.08 MiB |
| go_grpc | 14210 | 70.17 ms | 110.19 ms | 180.40 ms | 204.75 ms | 101.34% | 34.02 MiB |
| java_micronaut | 13819 | 72.23 ms | 98.14 ms | 105.78 ms | 187.33 ms | 100.99% | 452.57 MiB |
| java_micronaut_workstealing | 11958 | 83.48 ms | 101.84 ms | 110.44 ms | 188.77 ms | 101.22% | 470.4 MiB |
| node_grpc_st | 11591 | 86.12 ms | 96.10 ms | 101.19 ms | 108.65 ms | 100.1% | 48.04 MiB |
| swift_grpc | 11441 | 87.22 ms | 89.08 ms | 89.79 ms | 91.43 ms | 101.97% | 6.21 MiB |
| haskell_mu | 11221 | 88.97 ms | 106.24 ms | 111.58 ms | 125.49 ms | 101.41% | 125.47 MiB |
| scala_zio | 6136 | 162.62 ms | 216.79 ms | 296.44 ms | 386.12 ms | 100.74% | 265.08 MiB |
| node_grpcjs_st | 5400 | 184.75 ms | 200.41 ms | 204.80 ms | 220.67 ms | 101.03% | 43.0 MiB |
| csharp_grpc | 5356 | 186.27 ms | 210.86 ms | 216.79 ms | 272.85 ms | 101.79% | 104.96 MiB |
| elixir_grpc | 4376 | 227.96 ms | 257.66 ms | 265.14 ms | 284.74 ms | 100.43% | 88.04 MiB |
| scala_fs2 | 3018 | 330.28 ms | 383.57 ms | 407.74 ms | 514.12 ms | 102.24% | 222.9 MiB |
| python_grpc | 2173 | 458.14 ms | 488.93 ms | 494.38 ms | 506.48 ms | 102.29% | 33.9 MiB |
| ruby_grpc | 1959 | 507.92 ms | 574.11 ms | 596.82 ms | 681.75 ms | 97.46% | 31.7 MiB |
| java_aot | 1730 | 575.14 ms | 990.14 ms | 1.10 s | 1.40 s | 101.48% | 192.74 MiB |
| php_grpc | 1625 | 611.91 ms | 689.45 ms | 691.93 ms | 696.08 ms | 101.35% | 66.28 MiB |
| scala_akka | 1091 | 908.96 ms | 1.49 s | 1.70 s | 2.50 s | 100.57% | 263.46 MiB |
-----------------------------------------------------------------------------------------------------------------------------------------
2 CPU Server
-----------------------------------------------------------------------------------------------------------------------------------------
| name | req/s | avg. latency | 90 % in | 95 % in | 99 % in | avg. cpu | avg. memory |
-----------------------------------------------------------------------------------------------------------------------------------------
| java_hotspot_grpc_pgc | 70312 | 13.98 ms | 28.55 ms | 34.67 ms | 55.56 ms | 204.07% | 197.03 MiB |
| dotnet_grpc | 64515 | 15.33 ms | 20.46 ms | 23.09 ms | 29.85 ms | 198.9% | 131.58 MiB |
| cpp_grpc_mt | 63301 | 15.63 ms | 27.62 ms | 31.81 ms | 39.58 ms | 205.11% | 22.03 MiB |
| java_hotspot_grpc_g1gc | 62724 | 15.75 ms | 34.41 ms | 53.08 ms | 67.78 ms | 203.65% | 186.85 MiB |
| rust_tonic_mt | 61070 | 16.28 ms | 42.72 ms | 69.22 ms | 112.10 ms | 199.6% | 16.35 MiB |
| rust_grpcio | 56208 | 17.69 ms | 24.45 ms | 26.89 ms | 31.61 ms | 203.47% | 40.1 MiB |
| java_hotspot_grpc_zgc | 54091 | 18.29 ms | 38.62 ms | 50.15 ms | 67.14 ms | 204.23% | 120.16 MiB |
| java_openj9_grpc_gencon | 53745 | 18.47 ms | 43.49 ms | 50.35 ms | 66.15 ms | 204.2% | 142.71 MiB |
| java_hotspot_grpc_she | 53317 | 18.55 ms | 49.38 ms | 61.05 ms | 73.04 ms | 203.08% | 407.79 MiB |
| rust_thruster_mt | 52423 | 18.99 ms | 46.71 ms | 87.42 ms | 144.44 ms | 201.3% | 14.22 MiB |
| java_quarkus | 49270 | 20.16 ms | 41.30 ms | 44.91 ms | 52.74 ms | 163.39% | 117.09 MiB |
| java_hotspot_grpc_sgc | 48948 | 20.30 ms | 37.07 ms | 42.35 ms | 56.45 ms | 198.11% | 97.43 MiB |
| rust_tonic_st | 48826 | 20.41 ms | 24.81 ms | 26.04 ms | 28.78 ms | 99.63% | 14.79 MiB |
| kotlin_grpc | 46151 | 21.44 ms | 68.03 ms | 74.99 ms | 84.56 ms | 204.4% | 225.61 MiB |
| rust_thruster_st | 41577 | 23.98 ms | 29.05 ms | 30.52 ms | 32.85 ms | 101.36% | 12.26 MiB |
| go_grpc | 39447 | 25.14 ms | 65.68 ms | 72.96 ms | 86.92 ms | 193.03% | 37.01 MiB |
| cpp_grpc_st | 30323 | 32.91 ms | 36.60 ms | 37.91 ms | 39.95 ms | 102.72% | 8.77 MiB |
| java_micronaut | 22430 | 44.45 ms | 69.11 ms | 75.84 ms | 86.51 ms | 169.86% | 507.94 MiB |
| crystal_grpc | 22209 | 44.91 ms | 52.62 ms | 55.20 ms | 59.10 ms | 108.95% | 33.25 MiB |
| java_micronaut_workstealing | 21824 | 45.69 ms | 68.41 ms | 75.57 ms | 86.97 ms | 173.04% | 497.34 MiB |
| swift_grpc | 21347 | 46.74 ms | 49.27 ms | 50.37 ms | 52.55 ms | 201.42% | 6.63 MiB |
| csharp_grpc | 18528 | 53.85 ms | 85.00 ms | 89.63 ms | 96.33 ms | 204.51% | 108.98 MiB |
| haskell_mu | 18142 | 55.02 ms | 77.88 ms | 83.08 ms | 93.78 ms | 203.63% | 176.66 MiB |
| node_grpc_st | 11951 | 83.50 ms | 90.63 ms | 98.04 ms | 102.50 ms | 103.35% | 51.83 MiB |
| scala_zio | 11007 | 90.67 ms | 125.18 ms | 187.12 ms | 295.88 ms | 201.39% | 284.13 MiB |
| scala_fs2 | 10447 | 95.37 ms | 74.76 ms | 80.73 ms | 92.50 ms | 205.8% | 241.96 MiB |
| elixir_grpc | 8187 | 121.93 ms | 142.18 ms | 148.69 ms | 162.26 ms | 203.92% | 106.68 MiB |
| node_grpcjs_st | 7059 | 141.38 ms | 152.78 ms | 155.77 ms | 162.44 ms | 115.95% | 44.75 MiB |
| java_aot | 6883 | 144.98 ms | 278.06 ms | 305.32 ms | 399.25 ms | 194.85% | 227.25 MiB |
| scala_akka | 6117 | 163.08 ms | 301.52 ms | 397.43 ms | 598.65 ms | 201.83% | 266.85 MiB |
| python_grpc | 3996 | 249.61 ms | 278.59 ms | 282.89 ms | 292.29 ms | 150.71% | 33.98 MiB |
| php_grpc | 3989 | 250.01 ms | 288.64 ms | 289.96 ms | 292.68 ms | 205.37% | 67.51 MiB |
| ruby_grpc | 2550 | 390.55 ms | 420.60 ms | 432.11 ms | 475.37 ms | 131.24% | 31.87 MiB |
-----------------------------------------------------------------------------------------------------------------------------------------
3 CPU Server
-----------------------------------------------------------------------------------------------------------------------------------------
| name | req/s | avg. latency | 90 % in | 95 % in | 99 % in | avg. cpu | avg. memory |
-----------------------------------------------------------------------------------------------------------------------------------------
| java_hotspot_grpc_pgc | 85205 | 11.38 ms | 23.41 ms | 31.38 ms | 47.24 ms | 252.99% | 219.9 MiB |
| cpp_grpc_mt | 84014 | 11.57 ms | 21.42 ms | 27.69 ms | 43.96 ms | 263.92% | 39.0 MiB |
| java_hotspot_grpc_g1gc | 82253 | 11.83 ms | 24.06 ms | 31.79 ms | 47.09 ms | 271.86% | 186.95 MiB |
| java_openj9_grpc_gencon | 81968 | 11.85 ms | 22.23 ms | 29.01 ms | 46.69 ms | 283.63% | 144.94 MiB |
| java_hotspot_grpc_zgc | 80691 | 12.05 ms | 24.44 ms | 33.02 ms | 51.17 ms | 286.94% | 129.76 MiB |
| java_hotspot_grpc_she | 80457 | 12.09 ms | 27.43 ms | 38.36 ms | 56.73 ms | 280.09% | 427.27 MiB |
| go_grpc | 79458 | 12.34 ms | 22.42 ms | 26.87 ms | 36.35 ms | 305.78% | 35.24 MiB |
| rust_grpcio | 78670 | 12.20 ms | 21.67 ms | 28.35 ms | 48.47 ms | 271.5% | 37.29 MiB |
| rust_tonic_mt | 78592 | 12.44 ms | 23.99 ms | 31.50 ms | 48.51 ms | 280.44% | 16.72 MiB |
| dotnet_grpc | 77338 | 12.72 ms | 18.90 ms | 22.42 ms | 31.63 ms | 243.27% | 122.11 MiB |
| kotlin_grpc | 76864 | 12.71 ms | 26.44 ms | 35.53 ms | 50.08 ms | 291.95% | 227.84 MiB |
| rust_thruster_mt | 69965 | 14.06 ms | 27.15 ms | 35.84 ms | 54.57 ms | 289.27% | 14.53 MiB |
| java_hotspot_grpc_sgc | 61568 | 16.01 ms | 30.65 ms | 36.61 ms | 48.17 ms | 232.97% | 103.56 MiB |
| java_quarkus | 60937 | 16.19 ms | 29.82 ms | 41.91 ms | 53.87 ms | 204.78% | 106.03 MiB |
| rust_tonic_st | 49392 | 20.18 ms | 24.49 ms | 25.75 ms | 28.55 ms | 100.92% | 15.47 MiB |
| rust_thruster_st | 41856 | 23.82 ms | 28.21 ms | 29.50 ms | 31.65 ms | 100.55% | 12.17 MiB |
| csharp_grpc | 36534 | 27.24 ms | 50.19 ms | 55.02 ms | 62.67 ms | 307.14% | 110.34 MiB |
| java_micronaut | 30484 | 32.64 ms | 46.00 ms | 53.79 ms | 66.14 ms | 227.42% | 511.61 MiB |
| swift_grpc | 30274 | 32.95 ms | 35.62 ms | 36.40 ms | 37.88 ms | 302.68% | 6.88 MiB |
| cpp_grpc_st | 29923 | 33.35 ms | 36.41 ms | 37.84 ms | 39.66 ms | 102.35% | 8.36 MiB |
| java_micronaut_workstealing | 27721 | 26.60 ms | 48.96 ms | 58.17 ms | 71.11 ms | 212.94% | 511.27 MiB |
| haskell_mu | 23870 | 41.80 ms | 65.86 ms | 72.07 ms | 81.84 ms | 303.53% | 221.0 MiB |
| crystal_grpc | 22063 | 45.21 ms | 52.70 ms | 55.47 ms | 59.24 ms | 112.61% | 32.11 MiB |
| scala_zio | 17323 | 57.60 ms | 89.43 ms | 108.11 ms | 169.65 ms | 308.16% | 301.54 MiB |
| scala_akka | 14117 | 70.62 ms | 114.41 ms | 178.17 ms | 276.65 ms | 303.74% | 262.81 MiB |
| scala_fs2 | 13422 | 74.33 ms | 84.60 ms | 88.81 ms | 106.25 ms | 318.83% | 235.28 MiB |
| java_aot | 13047 | 76.49 ms | 158.10 ms | 184.81 ms | 245.73 ms | 280.2% | 233.64 MiB |
| node_grpc_st | 12417 | 80.38 ms | 87.73 ms | 94.16 ms | 99.30 ms | 103.61% | 47.94 MiB |
| elixir_grpc | 11728 | 85.12 ms | 101.03 ms | 106.86 ms | 119.49 ms | 306.5% | 102.65 MiB |
| node_grpcjs_st | 6966 | 143.27 ms | 154.24 ms | 156.55 ms | 163.74 ms | 115.38% | 43.67 MiB |
| php_grpc | 6624 | 150.66 ms | 181.26 ms | 182.57 ms | 185.51 ms | 307.94% | 65.71 MiB |
| python_grpc | 4033 | 247.32 ms | 273.18 ms | 276.77 ms | 285.34 ms | 151.67% | 34.15 MiB |
| ruby_grpc | 2581 | 385.86 ms | 416.70 ms | 427.30 ms | 467.66 ms | 131.68% | 33.0 MiB |
-----------------------------------------------------------------------------------------------------------------------------------------