Update on the development branch

[Shixiaowei02]

Hi,

The TensorRT-LLM team is pleased to announce that we have pushed an update to the development branch (and the Triton backend) this Dec 4, 2024.

This update includes:

• Model Support
  • Supported AutoAWQ and AutoGPTQ Hugging Face checkpoints for LLaMA. (Is it possible load quantized model from huggingface? #2458)
• Features
  • Supported guided decoding with XGrammar backend.
  • Supported typical acceptance for EAGLE.
  • Supported chunked context and sliding window attention to be enabled together.
  • Supported ulysses context parallel. Refer to an example on building LLaMA 7B using 2-way tensor parallelism and 2-way context parallelism at examples/llama/README.md.
  • Supported head size 64 for the XQA kernel.
  • Enabled unified converter for EAGLE checkpoint conversion.
  • [BREAKING CHANGE] Removed NVIDIA V100 GPU support.
  • [BREAKING CHANGE] Enforce keywords in model_runner.py from_dir.
• API
  • [BREAKING CHANGE] Remove enable_xqa argument from trtllm-build.
• Bug fixes
  • Fixed a typo in the error message. (Fix minor typo #2473)
  • Fix the issue where enable_context_fmha_fp32_acc was not being passed in the Python runtime.

Thanks,
The TensorRT-LLM Engineering Team