scx_layered: Add per layer timeslice

[hodgesds]

Allow setting a different timeslice per layer.

tested by adding some debug logs:

diff --git a/scheds/rust/scx_layered/src/bpf/main.bpf.c b/scheds/rust/scx_layered/src/bpf/main.bpf.c
index 14e6fc9..ebf3362 100644
--- a/scheds/rust/scx_layered/src/bpf/main.bpf.c
+++ b/scheds/rust/scx_layered/src/bpf/main.bpf.c
@@ -1230,20 +1230,21 @@ void BPF_STRUCT_OPS(layered_running, struct task_struct *p)
        if (tctx->last_cpu >= 0 && tctx->last_cpu != task_cpu)
                lstat_inc(LSTAT_MIGRATION, layer, cctx);
        tctx->last_cpu = task_cpu;
 
        if (vtime_before(layer->vtime_now, p->scx.dsq_vtime))
                layer->vtime_now = p->scx.dsq_vtime;
 
        cctx->current_preempt = layer->preempt;
        cctx->current_exclusive = layer->exclusive;
        tctx->running_at = bpf_ktime_get_ns();
+       dbg("CFG: layer: %d slice: %llu", layer->idx, layer->slice_ns);
 
        /*
         * If this CPU is transitioning from running an exclusive task to a
         * non-exclusive one, the sibling CPU has likely been idle. Wake it up.
         */
        if (cctx->prev_exclusive && !cctx->current_exclusive) {
                s32 sib = sibling_cpu(task_cpu);
                struct cpu_ctx *sib_cctx;
 
                /*

output:

   stress-ng-cpu-3306897 [023] dN..1 434025.244529: bpf_trace_printk: CFG: layer: 2 slice: 4000000
   stress-ng-cpu-3306898 [045] dN..1 434025.244529: bpf_trace_printk: CFG: layer: 2 slice: 4000000
   stress-ng-cpu-3306903 [074] dN..1 434025.244529: bpf_trace_printk: CFG: layer: 2 slice: 4000000
   grep-3306226 [051] dN..1 434025.244529: bpf_trace_printk: CFG: layer: 1 slice: 1500000

[hodgesds]

I guess this conversion doesn't need to be done since it already gets converted above...

[arighi]

You may also want to use layer_slice_ns to refill p->scx.slice if the task keeps running. Otherwise the task's time slice is automatically refilled to SCX_SLICE_DFL if I'm not wrong.

Maybe something like this (untested):

diff --git a/scheds/rust/scx_layered/src/bpf/main.bpf.c b/scheds/rust/scx_layered/src/bpf/main.bpf.c
index 14e6fc9..cb9ccbe 100644
--- a/scheds/rust/scx_layered/src/bpf/main.bpf.c
+++ b/scheds/rust/scx_layered/src/bpf/main.bpf.c
@@ -817,6 +817,7 @@ static bool keep_running(struct cpu_ctx *cctx, struct task_struct *p)
 		 */
 		if (disable_topology) {
 			if (!scx_bpf_dsq_nr_queued(layer->idx)) {
+				p->scx.slice = layer_slice_ns;
 				lstat_inc(LSTAT_KEEP, layer, cctx);
 				return true;
 			}
@@ -825,6 +826,7 @@ static bool keep_running(struct cpu_ctx *cctx, struct task_struct *p)
 						   tctx->last_cpu :
 						   bpf_get_smp_processor_id());
 			if (!scx_bpf_dsq_nr_queued(dsq_id)) {
+				p->scx.slice = layer_slice_ns;
 				lstat_inc(LSTAT_KEEP, layer, cctx);
 				return true;
 			}
@@ -850,6 +852,7 @@ static bool keep_running(struct cpu_ctx *cctx, struct task_struct *p)
 		scx_bpf_put_idle_cpumask(idle_cpumask);
 
 		if (has_idle) {
+			p->scx.slice = layer_slice_ns;
 			lstat_inc(LSTAT_KEEP, layer, cctx);
 			return true;
 		}

[hodgesds]

I think that worked alright, I spent a bit of time trying tuning different layer slice lengths and it's rather interesting to see how the slice can affect throughput vs latency.