Skip to content

Update default max_num_batch_tokens for chunked prefill#11694

Merged
mgoin merged 1 commit intovllm-project:mainfrom SachinVarghese:performance-docsJan 3, 2025

Commits

Commits on Jan 2, 2025