Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use instance pool allocation strategy for wasmtime shim #751

Merged
merged 4 commits into from
Nov 26, 2024

Conversation

andreiltd
Copy link
Member

@andreiltd andreiltd commented Nov 25, 2024

This patch changes wasm instance allocation strategy from OnDemand to PoolAllocator which increase a performance of creating a per request instances for components targeting wasi/http world.

Here, before and after throughput, for a minimum handler that responds with Hello World, shows 10x improvement:

Before:

hey -n 10000 -c 10 http://127.0.0.1:8080

Summary:
  Total:        2.6978 secs
  Slowest:      0.0135 secs
  Fastest:      0.0007 secs
  Average:      0.0027 secs
  Requests/sec: 3706.6977


Response time histogram:
  0.001 [1]     |
  0.002 [1323]  |■■■■■■■■
  0.003 [6765]  |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.005 [1769]  |■■■■■■■■■■
  0.006 [123]   |■
  0.007 [7]     |
  0.008 [1]     |
  0.010 [4]     |
  0.011 [4]     |
  0.012 [2]     |
  0.013 [1]     |


Latency distribution:
  10% in 0.0019 secs
  25% in 0.0022 secs
  50% in 0.0026 secs
  75% in 0.0031 secs
  90% in 0.0036 secs
  95% in 0.0039 secs
  99% in 0.0047 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0000 secs, 0.0007 secs, 0.0135 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0000 secs, 0.0000 secs, 0.0005 secs
  resp wait:    0.0026 secs, 0.0006 secs, 0.0123 secs
  resp read:    0.0000 secs, 0.0000 secs, 0.0009 secs

Status code distribution:
  [200] 10000 responses

After:

hey -n 10000 -c 10 http://127.0.0.1:8080

Summary:
  Total:        0.2566 secs
  Slowest:      0.0017 secs
  Fastest:      0.0001 secs
  Average:      0.0003 secs
  Requests/sec: 38975.5182
  

Response time histogram:
  0.000 [1]     |
  0.000 [5736]  |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.000 [3507]  |■■■■■■■■■■■■■■■■■■■■■■■■
  0.001 [478]   |■■■
  0.001 [159]   |■
  0.001 [72]    |■
  0.001 [28]    |
  0.001 [10]    |
  0.001 [4]     |
  0.002 [2]     |
  0.002 [3]     |


Latency distribution:
  10% in 0.0001 secs
  25% in 0.0002 secs
  50% in 0.0002 secs
  75% in 0.0003 secs
  90% in 0.0004 secs
  95% in 0.0005 secs
  99% in 0.0008 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0000 secs, 0.0001 secs, 0.0017 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0000 secs, 0.0000 secs, 0.0014 secs
  resp wait:    0.0002 secs, 0.0001 secs, 0.0012 secs
  resp read:    0.0000 secs, 0.0000 secs, 0.0009 secs

Status code distribution:
  [200] 10000 responses

Closes: #729

@andreiltd andreiltd added the wasmtime Anything wasmtime shim related label Nov 25, 2024
jprendes
jprendes previously approved these changes Nov 25, 2024
Copy link
Collaborator

@jprendes jprendes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
Just some minor comment.

crates/containerd-shim-wasmtime/src/instance.rs Outdated Show resolved Hide resolved
crates/containerd-shim-wasmtime/src/instance.rs Outdated Show resolved Hide resolved
Copy link
Collaborator

@jprendes jprendes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Mossaka Mossaka merged commit 75cc369 into containerd:main Nov 26, 2024
71 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wasmtime Anything wasmtime shim related
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Compare the throughput between the Wasmtime shim and wasmtime serve
3 participants