Code

[TPU][Quantization] TPU W8A8#11785

Code

Merged

robertgshaw2-redhat merged 73 commits intovllm-project:mainvllm-project/vllm:mainfrom neuralmagic:tpu-w8a8neuralmagic/vllm:tpu-w8a8Copy head branch name to clipboardJan 8, 2025

+565-190

Commits

Commits on Oct 11, 2024

• w8a8 working

  robertgshaw2-redhat

  committed
  3b0c8a6

  Copy full SHA for 3b0c8a6

• format

  robertgshaw2-redhat

  committed
  36fc1db

  Copy full SHA for 36fc1db

• added all kernels

  robertgshaw2-redhat

  committed
  d83c04c

  Copy full SHA for d83c04c

• format

  robertgshaw2-redhat

  committed
  af9d0f4

  Copy full SHA for af9d0f4

Commits on Oct 12, 2024

• working on cuda

  robertgshaw2-redhat

  committed
  0f9fd21

  Copy full SHA for 0f9fd21

• added mixed precision directory

  robertgshaw2-redhat

  committed
  7b3203f

  Copy full SHA for 7b3203f

• formatting

  robertgshaw2-redhat

  committed
  bf50fa4

  Copy full SHA for bf50fa4

• cache current state - w8a16 running oom

  robertgshaw2-redhat

  committed
  226ef52

  Copy full SHA for 226ef52

Commits on Oct 16, 2024

• [TPU] Ensure torch._sync(param) is called after param.data.copy_()

  WoosukKwon

  committed
  bb7c741

  Copy full SHA for bb7c741

Commits on Oct 17, 2024

• yapf

  WoosukKwon

  committed
  cf842bd

  Copy full SHA for cf842bd

• [TPU] Correctly profile peak memory usage

  WoosukKwon

  committed
  67039bc

  Copy full SHA for 67039bc

• Upgrade PyTorch XLA

  WoosukKwon

  committed
  0695f77

  Copy full SHA for 0695f77

• Merge branch 'main' into tpu-peak-mem

  WoosukKwon

  committed
  11cf82f

  Copy full SHA for 11cf82f

Commits on Oct 20, 2024

• stash

  robertgshaw2-redhat

  committed
  e016e38

  Copy full SHA for e016e38

• Merge branch 'main' into compressed-tensors-tpu

  robertgshaw2-redhat

  committed
  717b859

  Copy full SHA for 717b859

• proper merge

  robertgshaw2-redhat

  committed
  c848735

  Copy full SHA for c848735

• add mixed precision

  robertgshaw2-redhat

  committed
  1539915

  Copy full SHA for 1539915

• format

  robertgshaw2-redhat

  committed
  f00412a

  Copy full SHA for f00412a

• stash

  robertgshaw2-redhat

  committed
  b0a6b70

  Copy full SHA for b0a6b70

• Merge branch 'tpu-peak-mem' into compressed-tensors-tpu

  robertgshaw2-redhat

  committed
  e812d7e

  Copy full SHA for e812d7e

• stash

  robertgshaw2-redhat

  committed
  764dda1

  Copy full SHA for 764dda1

• remove name

  robertgshaw2-redhat

  committed
  87b2ae6

  Copy full SHA for 87b2ae6

• revert woosuk change

  robertgshaw2-redhat

  committed
  e813ff8

  Copy full SHA for e813ff8

• format

  robertgshaw2-redhat

  committed
  8cfaa1b

  Copy full SHA for 8cfaa1b

Commits on Oct 21, 2024

• update

  robertgshaw2-redhat

  committed
  bbc9741

  Copy full SHA for bbc9741

• fix nit

  robertgshaw2-redhat

  committed
  eb3f39e

  Copy full SHA for eb3f39e

• update

  robertgshaw2-redhat

  committed
  bb2fbe1

  Copy full SHA for bb2fbe1

• fix spurious

  robertgshaw2-redhat

  committed
  14ccb90

  Copy full SHA for 14ccb90

Commits on Oct 23, 2024

• stash branch for brittany

  robertgshaw2-redhat

  committed
  4092be2

  Copy full SHA for 4092be2

Commits on Jan 6, 2025

• Merge branch 'main' into tpu-w8a8

  robertgshaw2-redhat

  committed
  1aaa628

  Copy full SHA for 1aaa628

Commits on Jan 7, 2025

• revert

  robertgshaw2-redhat

  committed
  48aa54b

  Copy full SHA for 48aa54b

• fix

  robertgshaw2-redhat

  committed
  4efe915

  Copy full SHA for 4efe915

• updated

  robertgshaw2-redhat

  committed
  e98b79c

  Copy full SHA for e98b79c

• reduce cruft

  robertgshaw2-redhat

  committed
  5a89668

  Copy full SHA for 5a89668

• reduce cruft

  robertgshaw2-redhat

  committed
  57cbf5c

  Copy full SHA for 57cbf5c

• updated

  robertgshaw2-redhat

  committed
  3451c4d

  Copy full SHA for 3451c4d

• update comment

  robertgshaw2-redhat

  committed
  0c2e62a

  Copy full SHA for 0c2e62a

• revert spurious change

  robertgshaw2-redhat

  committed
  172c9ca

  Copy full SHA for 172c9ca

• remove cruft

  robertgshaw2-redhat

  committed
  938ca81

  Copy full SHA for 938ca81

• cruft reduction

  robertgshaw2-redhat

  committed
  9e18911

  Copy full SHA for 9e18911

• update docs

  robertgshaw2-redhat

  committed
  5f58ec7

  Copy full SHA for 5f58ec7

• added integration test

  robertgshaw2-redhat

  committed
  af9f298

  Copy full SHA for af9f298

• updated

  robertgshaw2-redhat

  committed
  6fe2f62

  Copy full SHA for 6fe2f62

• Add bias back

  robertgshaw2-redhat

  committed
  f2c0beb

  Copy full SHA for f2c0beb

• add bias support

  robertgshaw2-redhat

  committed
  8b29718

  Copy full SHA for 8b29718

• updated

  robertgshaw2-redhat

  committed
  1e2a373

  Copy full SHA for 1e2a373

• stash

  robertgshaw2-redhat

  committed
  2a359ef

  Copy full SHA for 2a359ef

• Merge branch 'main' into remove-async-stream

  robertgshaw2-redhat

  committed
  f7e8975

  Copy full SHA for f7e8975

• fix

  robertgshaw2-redhat

  committed
  0d4c3fd

  Copy full SHA for 0d4c3fd

• update

  robertgshaw2-redhat

  committed
  57340d2

  Copy full SHA for 57340d2

• trigger test in CI

  robertgshaw2-redhat

  committed
  38291d5

  Copy full SHA for 38291d5

• fix AZP

  robertgshaw2-redhat

  committed
  ead1e94

  Copy full SHA for ead1e94

• fixed!

  robertgshaw2-redhat

  committed
  cea5e54

  Copy full SHA for cea5e54

• Merge branch 'tpu-w8a8' of https://github.com/neuralmagic/vllm into tpu-w8a8

  robertgshaw2-redhat

  committed
  940ddde

  Copy full SHA for 940ddde

• fix azp adju

  robertgshaw2-redhat

  committed
  84a5b29

  Copy full SHA for 84a5b29

• make docker command look better on gh

  robertgshaw2-redhat

  committed
  a1d7b4a

  Copy full SHA for a1d7b4a

• remove torch warnings

  robertgshaw2-redhat

  committed
  2b4ecfd

  Copy full SHA for 2b4ecfd

• stash

  robertgshaw2-redhat

  committed
  186c108

  Copy full SHA for 186c108

• Merge branch 'tpu-w8a8' of https://github.com/neuralmagic/vllm into tpu-w8a8

  robertgshaw2-redhat

  committed
  7e8598a

  Copy full SHA for 7e8598a

• fix AZP

  robertgshaw2-redhat

  committed
  de773cd

  Copy full SHA for de773cd

• merged

  robertgshaw2-redhat

  committed
  3a53d7d

  Copy full SHA for 3a53d7d

• added

  robertgshaw2-redhat

  committed
  0be5f69

  Copy full SHA for 0be5f69

• fix formatting

  robertgshaw2-redhat

  committed
  cb69ba7

  Copy full SHA for cb69ba7

• remove comment

  robertgshaw2-redhat

  committed
  3896f6c

  Copy full SHA for 3896f6c

• formatted

  robertgshaw2-redhat

  committed
  33e1e13

  Copy full SHA for 33e1e13

• add llama to ci

  robertgshaw2-redhat

  committed
  dde72d6

  Copy full SHA for dde72d6

• Merge branch 'main' into tpu-w8a8

  robertgshaw2-redhat

  committed
  d7a9c93

  Copy full SHA for d7a9c93

• Update supported_hardware.md

  robertgshaw2-redhat

  authored
  db9f795

  Copy full SHA for db9f795

• Update supported_hardware.md

  robertgshaw2-redhat

  authored
  09ad869

  Copy full SHA for 09ad869

Commits on Jan 8, 2025

• ixed docs build

  robertgshaw2-redhat

  committed
  b74c88a

  Copy full SHA for b74c88a

• Merge branch 'tpu-w8a8' of https://github.com/neuralmagic/vllm into tpu-w8a8

  robertgshaw2-redhat

  committed
  da4369e

  Copy full SHA for da4369e

• Merge branch 'main' into tpu-w8a8

  robertgshaw2-redhat

  committed
  5ddcac2

  Copy full SHA for 5ddcac2

• fix CI

  robertgshaw2-redhat

  committed
  f353c43

  Copy full SHA for f353c43