feat: support custom task runner

[zhyncs]

Motivation

Usually, when we run experiments (benchmark or evaluation), we need to do it multiple times, manually starting the server cmd and client cmd each time and collecting results. This work is very procedural and tedious, so it can be semi-automated using scripts. cc @shanyu-sys @yichuan520030910320

Here is a simple example. ref https://github.com/sgl-project/sglang/actions/runs/12224157302

Modifications

Checklist

• Format your code according to the Contributor Guide.
• Add unit tests as outlined in the Contributor Guide.
• Update documentation as needed, including docstrings or example tutorials.

[zhyncs]

ref https://github.com/sgl-project/sglang/actions/runs/12224251746

[zhyncs]

TODO(zhyncs):
Make different config.yml files as CI input choices, so we can just write yml and manually trigger the execution of a specific config.
In theory, any group and any server cmd and client cmd can be used. The only things that need to be handled are the dependency installation for the server and the output formatting for the client.
For example, lm_eval, evalplus, etc., and we may want to run different configurations each time. This configurability gives us great flexibility without the need to set up a development environment ourselves.

[zhyncs]

In some cases, we only need to display, in other cases, we need to set a threshold for comparison.