Code

[WIP] Deepseek V2 MLA#10927

Code

Draft

simon-mo wants to merge 16 commits intovllm-project:mainvllm-project/vllm:mainfrom simon-mo:mlasimon-mo/vllm:mlaCopy head branch name to clipboard

+1,232-72

Commits

Commits on Dec 4, 2024

• Annotated MHA

  simon-mo

  committed
  95f1991

  Copy full SHA for 95f1991

Commits on Dec 5, 2024

• wip, cached latents, used mla decode, but it generate gibberish

  simon-mo

  committed
  e7a56da

  Copy full SHA for e7a56da

Commits on Dec 6, 2024

• impl vannilla prefill w/kv cache, now it generate with matabsob, debugging fi mla kernel issue

  simon-mo

  committed
  5bb7586

  Copy full SHA for 5bb7586

• lint

  simon-mo

  committed
  feb6ba3

  Copy full SHA for feb6ba3

• eager mode works

  simon-mo

  committed
  f585a3f

  Copy full SHA for f585a3f

Commits on Dec 25, 2024

• reformat and add FLASHINFER_MLA

  simon-mo

  committed
  611acaa

  Copy full SHA for 611acaa

• moved prefill FA into FLASHINFER_MLA backend

  simon-mo

  committed
  2ccc97d

  Copy full SHA for 2ccc97d

Commits on Dec 31, 2024

• working TP?

  Show description for 7b92036
  simon-mo

  committed
  7b92036

  Copy full SHA for 7b92036

• feature flag working

  Show description for b8f63d7
  simon-mo

  committed
  b8f63d7

  Copy full SHA for b8f63d7

• add env flag

  Show description for 043085d
  simon-mo

  committed
  043085d

  Copy full SHA for 043085d

Commits on Jan 2, 2025

• Merge branch 'main' of github.com:vllm-project/vllm into mla

  simon-mo

  committed
  19167ca

  Copy full SHA for 19167ca

• lint

  Show description for 6ea2c06
  simon-mo

  committed
  6ea2c06

  Copy full SHA for 6ea2c06

Commits on Jan 6, 2025

• more debug lines

  Show description for f668cb9
  simon-mo

  committed
  f668cb9

  Copy full SHA for f668cb9

Commits on Jan 16, 2025

• Use yarn scales, fixes some accuracy issue

  Show description for 4094e2d
  

  simon-mo

  and

  cennn

  committed
  4094e2d

  Copy full SHA for 4094e2d

• format

  Show description for a047772
  simon-mo

  committed
  a047772

  Copy full SHA for a047772

• Merge branch 'main' of github.com:vllm-project/vllm into mla

  simon-mo

  committed
  784a5a4

  Copy full SHA for 784a5a4