scx_layered: Add stats for XNUMA/XLLC migrations #648
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hodgesds
commented
Sep 11, 2024
| s32 task_cpu = scx_bpf_task_cpu(p); | ||
|
|
||
| if (!(cctx = lookup_cpu_ctx(-1)) || !(tctx = lookup_task_ctx(p)) || | ||
| !(layer = lookup_layer(tctx->layer))) | ||
| return; | ||
|
|
||
| if (tctx->last_cpu >= 0 && tctx->last_cpu != task_cpu) | ||
| if (tctx->last_cpu >= 0 && tctx->last_cpu != task_cpu) { | ||
| if (!(nodec = lookup_node_ctx(cctx->node_idx))) |
There was a problem hiding this comment.
I'm only doing the lookup for the node_ctx/cache_ctx on migration, figure it saves a few lookups.
hodgesds
commented
Sep 11, 2024
| return ret; | ||
| } | ||
|
|
||
| static s32 create_cache(u32 cache_id) |
There was a problem hiding this comment.
This method probably should have existed for awhile, but I hadn't had a use for the cache_ctx yet.
Add stats for XNUMA/XLLC migrations. An example of the output is shown:
```
hodgesd : util/frac= 5.4/ 0.1 load/frac= 301.0/ 0.3 tasks= 476
tot= 3168 local=97.82 wake/exp/reenq= 2.18/ 0.00/ 0.00
keep/max/busy= 0.03/ 0.00/ 0.03 kick= 0.00 yield/ign= 0.09/ 0
open_idle= 0.00 mig= 6.82 xnuma_mig= 6.82 xllc_mig= 4.86 affn_viol= 0.00
preempt/first/idle/fail= 0.00/ 0.00/ 0.00/ 0.00 min_exec= 0.00/ 0.00ms
cpus= 2 [ 2, 4] 00000000 00000010 00001000
normal : util/frac= 28.7/ 0.7 load/frac= 101704.7/ 95.8 tasks= 2450
tot= 4660 local=99.06 wake/exp/reenq= 0.88/ 0.06/ 0.00
keep/max/busy= 1.03/ 0.00/ 0.00 kick= 0.06 yield/ign= 0.04/ 400
open_idle=15.73 mig=23.45 xnuma_mig=23.45 xllc_mig= 3.07 affn_viol= 0.00
preempt/first/idle/fail= 0.00/ 0.00/ 0.00/ 0.88 min_exec= 0.00/ 0.00ms
cpus= 2 [ 2, 2] 00000001 00000100 00000000
excl_coll=12.55 excl_preempt= 0.00
random : util/frac= 0.0/ 0.0 load/frac= 0.0/ 0.0 tasks= 0
tot= 0 local= 0.00 wake/exp/reenq= 0.00/ 0.00/ 0.00
keep/max/busy= 0.00/ 0.00/ 0.00 kick= 0.00 yield/ign= 0.00/ 0
open_idle= 0.00 mig= 0.00 xnuma_mig= 0.00 xllc_mig= 0.00 affn_viol= 0.00
preempt/first/idle/fail= 0.00/ 0.00/ 0.00/ 0.00 min_exec= 0.00/ 0.00ms
cpus= 0 [ 0, 0] 00000000 00000000 00000000
excl_coll= 0.00 excl_preempt= 0.00
stress-ng: util/frac= 4189.1/ 99.2 load/frac= 4200.0/ 4.0 tasks= 43
tot= 62 local= 0.00 wake/exp/reenq= 0.00/100.0/ 0.00
keep/max/busy=2433.9/177.4/ 0.00 kick=100.0 yield/ign= 3.23/ 0
open_idle= 0.00 mig=54.84 xnuma_mig=54.84 xllc_mig=35.48 affn_viol= 0.00
preempt/first/idle/fail= 0.00/ 0.00/ 0.00/ 0.00 min_exec= 0.00/ 0.00ms
cpus= 4 [ 4, 4] 00000300 00030000 00000000
excl_coll= 0.00 excl_preempt= 0.00
```
Signed-off-by: Daniel Hodges <[email protected]>
hodgesds
force-pushed
the
layered-llc-stats
branch
from
7025478
to
43ec8bf
Compare
hodgesds
commented
Sep 11, 2024
| @@ -147,6 +144,30 @@ struct { | |||
| __uint(map_flags, 0); | |||
| } node_data SEC(".maps"); | |||
|
|
|||
| static struct node_ctx *lookup_node_ctx(u32 node) | |||
There was a problem hiding this comment.
should probably make this node_idx to be consistent...
Initialize the node cpumask, which was previously uninitialized causing metric calculations to be wrong when attempting to lookup CPUs in the node cpumask. Signed-off-by: Daniel Hodges <[email protected]>
hodgesds
force-pushed
the
layered-llc-stats
branch
from
a93b19d
to
ae57f8d
Compare
|
Tested the NUMA mask with: and output: |
htejun
approved these changes
Sep 12, 2024
hodgesds
added a commit
to hodgesds/scx
that referenced
this pull request
Sep 12, 2024
scx_layered: Add stats for XNUMA/XLLC migrations
hodgesds
added a commit
to hodgesds/scx
that referenced
this pull request
Sep 12, 2024
scx_layered: Add stats for XNUMA/XLLC migrations
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add stats for XNUMA/XLLC migrations. An example of the output is shown:
This was tested on a machine with two NUMA nodes and two LLCs so the
xnuma_mig == xllc_mig.