Code

Upstream merge 25 01 13#358

Code

Merged

gshtras merged 116 commits intomainROCm/vllm:mainfrom upstream_merge_25_01_13ROCm/vllm:upstream_merge_25_01_13Copy head branch name to clipboardJan 14, 2025

+12,609-6,071

Commits

Commits on Jan 6, 2025

• [Bugfix][V1] Fix molmo text-only inputs (vllm-project#11676)

  Show description for 32c9eff
  jeejeelee

  authored
  32c9eff

  Copy full SHA for 32c9eff

• [Kernel] Move attn_type to Attention.__init__() (vllm-project#11690)

  Show description for e20c92b
  heheda12345

  authored
  e20c92b

  Copy full SHA for e20c92b

• [V1] Extend beyond image modality and support mixed-modality inference with Llava-OneVision (vllm-project#11685)

  Show description for 91b361a
  

  ywang96

  and

  DarkLight1337

  authored
  91b361a

  Copy full SHA for 91b361a

Commits on Jan 7, 2025

• [Bugfix] Fix LLaVA-NeXT feature size precision error (for real) (vllm-project#11772)

  Show description for 08fb75c
  DarkLight1337

  authored
  08fb75c

  Copy full SHA for 08fb75c

• [Model] Future-proof Qwen2-Audio multi-modal processor (vllm-project#11776)

  Show description for d0169e1
  DarkLight1337

  authored
  d0169e1

  Copy full SHA for d0169e1

• [XPU] Make pp group initilized for pipeline-parallelism (vllm-project#11648)

  Show description for d93d2d7
  ys950902

  authored
  d93d2d7

  Copy full SHA for d93d2d7

• [Doc][3/N] Reorganize Serving section (vllm-project#11766)

  Show description for 8ceffbf
  DarkLight1337

  authored
  8ceffbf

  Copy full SHA for 8ceffbf

• [Kernel][LoRA]Punica prefill kernels fusion (vllm-project#11234)

  Show description for b278557
  

  jeejeelee

  and

  Abatom

  authored
  b278557

  Copy full SHA for b278557

• [Bugfix] Update attention interface in Whisper (vllm-project#11784)

  Show description for 0f3f3c8
  ywang96

  authored
  0f3f3c8

  Copy full SHA for 0f3f3c8

• [CI] Fix neuron CI and run offline tests (vllm-project#11779)

  Show description for 898cdf0
  liangfu

  authored
  898cdf0

  Copy full SHA for 898cdf0

• fix init error for MessageQueue when n_local_reader is zero (vllm-project#11768)

  XiaobingSuper

  authored
  e512f76

  Copy full SHA for e512f76

• [Doc] Create a vulnerability management team (vllm-project#9925)

  Show description for ce1917f
  russellb

  authored
  ce1917f

  Copy full SHA for ce1917f

• [CI][CPU] adding build number to docker image name (vllm-project#11788)

  Show description for 1e4ce29
  zhouyuan

  authored
  1e4ce29

  Copy full SHA for 1e4ce29

• [V1][Doc] Update V1 support for LLaVa-NeXT-Video (vllm-project#11798)

  Show description for 8082ad7
  ywang96

  authored
  8082ad7

  Copy full SHA for 8082ad7

• [Bugfix] Comprehensively test and fix LLaVA-NeXT feature size calculation (vllm-project#11800)

  Show description for 8f37be3
  DarkLight1337

  authored
  8f37be3

  Copy full SHA for 8f37be3

• [doc] add doc to explain how to use uv (vllm-project#11773)

  Show description for 869e829
  

  youkaichao

  and

  DarkLight1337

  authored
  869e829

  Copy full SHA for 869e829

• [V1] Support audio language models on V1 (vllm-project#11733)

  Show description for 2de197b
  ywang96

  authored
  2de197b

  Copy full SHA for 2de197b

• [doc] update how pip can install nightly wheels (vllm-project#11806)

  Show description for d9fa1c0
  youkaichao

  authored
  d9fa1c0

  Copy full SHA for d9fa1c0

• [Doc] Add note to gte-Qwen2 models (vllm-project#11808)

  Show description for c0efe92
  DarkLight1337

  authored
  c0efe92

  Copy full SHA for c0efe92

• [optimization] remove python function call for custom op (vllm-project#11750)

  Show description for 869579a
  youkaichao

  authored
  869579a

  Copy full SHA for 869579a

• [Bugfix] update the prefix for qwen2 (vllm-project#11795)

  Show description for c994223
  

  jiangjiadi

  and

  jiadi.jjd

  authored
  c994223

  Copy full SHA for c994223

• [Doc]Add documentation for using EAGLE in vLLM (vllm-project#11417)

  Show description for 973f5dc
  sroy745

  authored
  973f5dc

  Copy full SHA for 973f5dc

Commits on Jan 8, 2025

• [Bugfix] Significant performance drop on CPUs with --num-scheduler-steps > 1 (vllm-project#11794)

  DamonFool

  authored
  a4e2b26

  Copy full SHA for a4e2b26

• [Doc] Group examples into categories (vllm-project#11782)

  Show description for 5950f55
  hmellor

  authored
  5950f55

  Copy full SHA for 5950f55

• [Bugfix] Fix image input for Pixtral-HF (vllm-project#11741)

  Show description for 91445c7
  DarkLight1337

  authored
  91445c7

  Copy full SHA for 91445c7

• [Misc] sort torch profiler table by kernel timing (vllm-project#11813)

  divakar-amd

  authored
  4d29e91

  Copy full SHA for 4d29e91

• Remove the duplicate imports of MultiModalKwargs and PlaceholderRange… (vllm-project#11824)

  WangErXiao

  authored
  dc71af0

  Copy full SHA for dc71af0

• Fixed docker build for ppc64le (vllm-project#11518)

  Show description for b640b19
  npanpaliya

  authored
  b640b19

  Copy full SHA for b640b19

• [OpenVINO] Fixed Docker.openvino build (vllm-project#11732)

  Show description for f4923cb
  ilya-lavrenov

  authored
  f4923cb

  Copy full SHA for f4923cb

• [Bugfix] Add checks for LoRA and CPU offload (vllm-project#11810)

  Show description for f645eb6
  jeejeelee

  authored
  f645eb6

  Copy full SHA for f645eb6

• [Docs] reorganize sponsorship page (vllm-project#11639)

  Show description for 259abd8
  simon-mo

  authored
  259abd8

  Copy full SHA for 259abd8

• [Bug] Fix pickling of ModelConfig when RunAI Model Streamer is used (vllm-project#11825)

  Show description for ef68eb2
  DarkLight1337

  authored
  ef68eb2

  Copy full SHA for ef68eb2

• [misc] improve memory profiling (vllm-project#11809)

  Show description for 889e662
  

  youkaichao

  and

  DarkLight1337

  authored
  889e662

  Copy full SHA for 889e662

• [doc] update wheels url (vllm-project#11830)

  Show description for ad9f1aa
  youkaichao

  authored
  ad9f1aa

  Copy full SHA for ad9f1aa

• [Docs] Update sponsor name: 'Novita' to 'Novita AI' (vllm-project#11833)

  simon-mo

  authored
  a1b2b86

  Copy full SHA for a1b2b86

• [Hardware][Apple] Native support for macOS Apple Silicon (vllm-project#11696)

  Show description for cfd3219
  

  wallashss

  and

  mgoin

  authored
  cfd3219

  Copy full SHA for cfd3219

• [torch.compile] consider relevant code in compilation cache (vllm-project#11614)

  Show description for f121411
  youkaichao

  authored
  f121411

  Copy full SHA for f121411

• [VLM] Reorganize profiling/processing-related code (vllm-project#11812)

  Show description for 2a0596b
  DarkLight1337

  authored
  2a0596b

  Copy full SHA for 2a0596b

• [Doc] Move examples into categories (vllm-project#11840)

  Show description for aba8d6e
  hmellor

  authored
  aba8d6e

  Copy full SHA for aba8d6e

• [Doc][4/N] Reorganize API Reference (vllm-project#11843)

  Show description for 6cd40a5
  DarkLight1337

  authored
  6cd40a5

  Copy full SHA for 6cd40a5

• [CI/Build][Bugfix] Fix CPU CI image clean up (vllm-project#11836)

  Show description for 2f70249
  bigPYJ1151

  authored
  2f70249

  Copy full SHA for 2f70249

• [Bugfix][XPU] fix silu_and_mul (vllm-project#11823)

  Show description for 78f4590
  yma11

  authored
  78f4590

  Copy full SHA for 78f4590

• [Misc] Move some model utils into vision file (vllm-project#11848)

  Show description for ca47e17
  DarkLight1337

  authored
  ca47e17

  Copy full SHA for ca47e17

• [Doc] Expand Multimodal API Reference (vllm-project#11852)

  Show description for 5984499
  DarkLight1337

  authored
  5984499

  Copy full SHA for 5984499

• [Misc]add some explanations for BlockHashType (vllm-project#11847)

  WangErXiao

  authored
  47de882

  Copy full SHA for 47de882

• [TPU][Quantization] TPU W8A8 (vllm-project#11785)

  Show description for 56fe4c2
  

  robertgshaw2-redhat

  and

  WoosukKwon

  authored
  56fe4c2

  Copy full SHA for 56fe4c2

• [Kernel][Triton][AMD] Use block size heuristic for avg 2.8x speedup for int8 models (vllm-project#11698)

  Show description for 526de82
  rasmith

  authored
  526de82

  Copy full SHA for 526de82

• [Docs] Add Google Cloud Meetup (vllm-project#11864)

  simon-mo

  authored
  3db0caf

  Copy full SHA for 3db0caf

Commits on Jan 9, 2025

• [CI] Turn on basic correctness tests for V1 (vllm-project#10864)

  tlrmchlsmth

  authored
  615e4a5

  Copy full SHA for 615e4a5

• treat do_lower_case in the same way as the sentence-transformers library (vllm-project#11815)

  Show description for 1fe554b
  maxdebayser

  authored
  1fe554b

  Copy full SHA for 1fe554b

• [Doc] Recommend uv and python 3.12 for quickstart guide (vllm-project#11849)

  Show description for 730e959
  mgoin

  authored
  730e959

  Copy full SHA for 730e959

• [Misc] Move print_*_once from utils to logger (vllm-project#11298)

  Show description for d848800
  

  DarkLight1337

  and

  mfournioux

  authored
  d848800

  Copy full SHA for d848800

• [Doc] Intended links Python multiprocessing library (vllm-project#11878)

  guspan-tanadi

  authored
  a732900

  Copy full SHA for a732900

• [perf]fix current stream (vllm-project#11870)

  Show description for 310aca8
  youkaichao

  authored
  310aca8

  Copy full SHA for 310aca8

• [Bugfix] Override dunder methods of placeholder modules (vllm-project#11882)

  Show description for 0bd1ff4
  DarkLight1337

  authored
  0bd1ff4

  Copy full SHA for 0bd1ff4

• [Bugfix] fix beam search input errors and latency benchmark script (vllm-project#11875)

  Show description for 1d967ac
  

  yeqcharlotte

  and

  yeq

  authored
  1d967ac

  Copy full SHA for 1d967ac

• [Doc] Add model development API Reference (vllm-project#11884)

  Show description for 65097ca
  DarkLight1337

  authored
  65097ca

  Copy full SHA for 65097ca

• [platform] Allow platform specify attention backend (vllm-project#11609)

  Show description for 405eb8e
  

  wangxiyuan

  and

  MengqingCao

  authored
  405eb8e

  Copy full SHA for 405eb8e

• [ci]try to fix flaky multi-step tests (vllm-project#11894)

  Show description for bd82872
  youkaichao

  authored
  bd82872

  Copy full SHA for bd82872

• [Misc] Provide correct Pixtral-HF chat template (vllm-project#11891)

  Show description for 9a22834
  DarkLight1337

  authored
  9a22834

  Copy full SHA for 9a22834

• [Docs] Add Modal to deployment frameworks (vllm-project#11907)

  charlesfrye

  authored
  36f5303

  Copy full SHA for 36f5303

Commits on Jan 10, 2025

• [Doc][5/N] Move Community and API Reference to the bottom (vllm-project#11896)

  Show description for c3cf54d
  

  DarkLight1337

  and

  simon-mo

  authored
  c3cf54d

  Copy full SHA for c3cf54d

• [VLM] Enable tokenized inputs for merged multi-modal processor (vllm-project#11900)

  Show description for b844b99
  DarkLight1337

  authored
  b844b99

  Copy full SHA for b844b99

• [Doc] Show default pooling method in a table (vllm-project#11904)

  Show description for 3de2b1e
  DarkLight1337

  authored
  3de2b1e

  Copy full SHA for 3de2b1e

• [torch.compile] Hide KV cache behind torch.compile boundary (vllm-project#11677)

  Show description for cf5f000
  heheda12345

  authored
  cf5f000

  Copy full SHA for cf5f000

• [Bugfix] Validate lora adapters to avoid crashing server (vllm-project#11727)

  Show description for ac2f3f7
  

  joerunde

  and

  jeejeelee

  authored
  ac2f3f7

  Copy full SHA for ac2f3f7

• [BUGFIX] Fix UnspecifiedPlatform package name (vllm-project#11916)

  Show description for 61af633
  jikunshang

  authored
  61af633

  Copy full SHA for 61af633

• [ci] fix gh200 tests (vllm-project#11919)

  Show description for d53575a
  youkaichao

  authored
  d53575a

  Copy full SHA for d53575a

• [misc] remove python function call for custom activation op (vllm-project#11885)

  Show description for d907be7
  

  cennn

  and

  youkaichao

  authored
  d907be7

  Copy full SHA for d907be7

• [platform] support pytorch custom op pluggable (vllm-project#11328)

  Show description for ef725fe
  wangxiyuan

  authored
  ef725fe

  Copy full SHA for ef725fe

• Replace "online inference" with "online serving" (vllm-project#11923)

  Show description for d85c47d
  hmellor

  authored
  d85c47d

  Copy full SHA for d85c47d

• [ci] Fix sampler tests (vllm-project#11922)

  Show description for 241ad7b
  youkaichao

  authored
  241ad7b

  Copy full SHA for 241ad7b

• [Doc] [1/N] Initial guide for merged multi-modal processor (vllm-project#11925)

  Show description for 12664dd
  DarkLight1337

  authored
  12664dd

  Copy full SHA for 12664dd

• [platform] support custom torch.compile backend key (vllm-project#11318)

  Show description for 20410b2
  

  wangxiyuan

  and

  youkaichao

  authored
  20410b2

  Copy full SHA for 20410b2

• [Doc] Rename offline inference examples (vllm-project#11927)

  Show description for 482cdc4
  hmellor

  authored
  482cdc4

  Copy full SHA for 482cdc4

• [Docs] Fix docstring in get_ip function (vllm-project#11932)

  Show description for f33e033
  KuntaiDu

  authored
  f33e033

  Copy full SHA for f33e033

• Doc fix in benchmark_long_document_qa_throughput.py (vllm-project#11933)

  Show description for 5959564
  KuntaiDu

  authored
  5959564

  Copy full SHA for 5959564

• [Hardware][CPU] Support MOE models on x86 CPU (vllm-project#11831)

  Show description for aa1e77a
  bigPYJ1151

  authored
  aa1e77a

  Copy full SHA for aa1e77a

• [Misc] Clean up debug code in Deepseek-V3 (vllm-project#11930)

  Show description for 46fa98c
  Isotr0py

  authored
  46fa98c

  Copy full SHA for 46fa98c

• [Misc] Update benchmark_prefix_caching.py fixed example usage (vllm-project#11920)

  Show description for 8a57940
  

  remimin

  and

  Ren MinMin

  authored
  8a57940

  Copy full SHA for 8a57940

• [Bugfix] Check that number of images matches number of <|image|> tokens with mllama (vllm-project#11939)

  Show description for d45cbe7
  tjohnson31415

  authored
  d45cbe7

  Copy full SHA for d45cbe7

Commits on Jan 11, 2025

• [mypy] Fix mypy warnings in api_server.py (vllm-project#11941)

  Show description for c9f09a4
  frreiss

  authored
  c9f09a4

  Copy full SHA for c9f09a4

• [ci] fix broken distributed-tests-4-gpus (vllm-project#11937)

  Show description for 899136b
  youkaichao

  authored
  899136b

  Copy full SHA for 899136b

• [Bugfix][SpecDecode] Adjust Eagle model architecture to align with intended design (vllm-project#11672)

  Show description for 2118d05
  llsj14

  authored
  2118d05

  Copy full SHA for 2118d05

• [Bugfix] fused_experts_impl wrong compute type for float32 (vllm-project#11921)

  Show description for c32a7c7
  

  shaochangxu

  and

  shaochangxu.scx

  authored
  c32a7c7

  Copy full SHA for c32a7c7

• [CI/Build] Move model-specific multi-modal processing tests (vllm-project#11934)

  Show description for 7a3a83e
  DarkLight1337

  authored
  7a3a83e

  Copy full SHA for 7a3a83e

• [Doc] Basic guide for writing unit tests for new models (vllm-project#11951)

  Show description for a991f7d
  DarkLight1337

  authored
  a991f7d

  Copy full SHA for a991f7d

• [Bugfix] Fix RobertaModel loading (vllm-project#11940)

  Show description for d697dc0
  NickLucche

  authored
  d697dc0

  Copy full SHA for d697dc0

• [Model] Add cogagent model support vLLM (vllm-project#11742)

  Show description for 4b657d3
  

  sixsixcoder

  and

  Isotr0py

  authored
  4b657d3

  Copy full SHA for 4b657d3

Commits on Jan 12, 2025

• [V1] Avoid sending text prompt to core engine (vllm-project#11963)

  Show description for b25cfab
  ywang96

  authored
  b25cfab

  Copy full SHA for b25cfab

• [CI/Build] Add markdown linter (vllm-project#11857)

  Show description for 43f3d9e
  rafvasq

  authored
  43f3d9e

  Copy full SHA for 43f3d9e

• [Model] Initialize support for Deepseek-VL2 models (vllm-project#11578)

  Show description for f967e51
  

  Isotr0py

  and

  DarkLight1337

  authored
  f967e51

  Copy full SHA for f967e51

• [Hardware][CPU] Multi-LoRA implementation for the CPU backend (vllm-project#11100)

  Show description for 8bddb73
  

  4 peopleauthored
  8bddb73

  Copy full SHA for 8bddb73

• [Hardware][TPU] workaround fix for MoE on TPU (vllm-project#11764)

  avshalomman

  authored
  263a870

  Copy full SHA for 263a870

• [V1][Core][1/n] Logging and Metrics (vllm-project#11962)

  Show description for 9597a09
  robertgshaw2-redhat

  authored
  9597a09

  Copy full SHA for 9597a09

Commits on Jan 13, 2025

• [Model] Support GGUF models newly added in transformers 4.46.0 (vllm-project#9685)

  Show description for d14e98d
  

  Isotr0py

  and

  DarkLight1337

  authored
  d14e98d

  Copy full SHA for d14e98d

• [V1] [2/n] Logging and Metrics - OutputProcessor Abstraction (vllm-project#11973)

  Show description for 619ae26
  robertgshaw2-redhat

  authored
  619ae26

  Copy full SHA for 619ae26

• [MISC] fix typo in kv transfer send recv test (vllm-project#11983)

  yyccli

  authored
  f7b3ba8

  Copy full SHA for f7b3ba8

• [Bug] Fix usage of .transpose() and .view() consecutively. (vllm-project#11979)

  liaoyanqing666

  authored
  9dd02d8

  Copy full SHA for 9dd02d8

• [CI][Spec Decode] fix: broken test for EAGLE model (vllm-project#11972)

  Show description for 80ea3af
  llsj14

  authored
  80ea3af

  Copy full SHA for 80ea3af

• [Misc] Fix Deepseek V2 fp8 kv-scale remapping (vllm-project#11947)

  Show description for cf6bbcb
  Concurrensee

  authored
  cf6bbcb

  Copy full SHA for cf6bbcb

• [Misc]Minor Changes about Worker (vllm-project#11555)

  Show description for c3f05b0
  noemotiovon

  authored
  c3f05b0

  Copy full SHA for c3f05b0

• [platform] add ray_device_key (vllm-project#11948)

  Show description for 89ce62a
  youkaichao

  authored
  89ce62a

  Copy full SHA for 89ce62a

• Fix Max Token ID for Qwen-VL-Chat (vllm-project#11980)

  Show description for 5340a30
  alex-jw-brooks

  authored
  5340a30

  Copy full SHA for 5340a30

• [Kernel] unified_attention for Attention.forward (vllm-project#11967)

  Show description for 0f8cafe
  heheda12345

  authored
  0f8cafe

  Copy full SHA for 0f8cafe

• [Doc][V1] Update model implementation guide for V1 support (vllm-project#11998)

  Show description for cd82499
  

  ywang96

  and

  DarkLight1337

  authored
  cd82499

  Copy full SHA for cd82499

• [Doc] Organise installation documentation into categories and tabs (vllm-project#11935)

  Show description for e8c23ff
  hmellor

  authored
  e8c23ff

  Copy full SHA for e8c23ff

• [platform] add device_control env var (vllm-project#12009)

  Show description for 458e63a
  youkaichao

  authored
  458e63a

  Copy full SHA for 458e63a

• [Platform] Move get_punica_wrapper() function to Platform (vllm-project#11516)

  Show description for a7d5968
  shen-shanshan

  authored
  a7d5968

  Copy full SHA for a7d5968

• bugfix: Fix signature mismatch in benchmark's get_tokenizer function (vllm-project#11982)

  Show description for c6db213
  e1ijah1

  authored
  c6db213

  Copy full SHA for c6db213

• Merge remote-tracking branch 'upstream/main'

  gshtras

  committed
  ce53f46

  Copy full SHA for ce53f46

• Using list

  gshtras

  committed
  5a51290

  Copy full SHA for 5a51290

• Revert "[misc] improve memory profiling (vllm-project#11809)"

  Show description for 079750e
  gshtras

  committed
  079750e

  Copy full SHA for 079750e

• Trying to make scales work with compileable attention

  gshtras

  committed
  043c93d

  Copy full SHA for 043c93d

Commits on Jan 14, 2025

• Docs lint

  gshtras

  committed
  16f8680

  Copy full SHA for 16f8680

• Merge remote-tracking branch 'origin/main' into upstream_merge_25_01_13

  gshtras

  committed
  eb4abfd

  Copy full SHA for eb4abfd