Fix the default chunked prefill size

[merrymercy]

This is a follow up for the chunked prefill size adjustment in #2225

• Only do chunk size adjustment when the chunked prefill size is not specified by the arguments, so we do not silently change users' arguments.
• Use a smaller cuda graph max bs for small memory GPUs because cuda graph typically does not bring speedup on these GPUs. A smaller cuda graph max bs saves memory and prevents OOM.

[merrymercy]

cc @BBuf

[BBuf]

cc @BBuf

Thanks. Actually, in my nsight-system profiling of Qwen2.5 inference on HuggingFace, I observed that using cuda-graph made no difference because the kernel launch time was already at nanosecond level. I'd like to know the reason for this. In contrast, on A800, without using cuda graph, the kernel launch time during decoding phase is even longer than the inference time itself. With cuda graph enabled, the time for a complete decoding iteration can be reduced by half.

[merrymercy]

@BBuf I do not fully understand that as well. Based on my e2e test. On A100/H100, cuda graph is very useful. On low-end GPUs (3090, A10G, L40), it almost has no effect when tp=1.

I do not have time to dig into that, but you can play with it more.