Skip to content

Issues: NVIDIA/TensorRT-LLM

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

545 Open 1,461 Closed
545 Open 1,461 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

qserve convert checkpoint raise an error bug Something isn't working
#2507 opened Nov 27, 2024 by anaivebird
2 of 4 tasks
1
Medusa quality impact + max_draft_len overhead impact
#2506 opened Nov 26, 2024 by ValeGian
1
How to visualize network?
#2501 opened Nov 26, 2024 by youki-sada
[QST] How to get the prefill latency and TPOT resepectly when using C++ runtime
#2500 opened Nov 26, 2024 by gujiewen
How to extract the hidden_states before output_ids during the inference process.
#2499 opened Nov 26, 2024 by PCJin2
Enabling custom MoE routing distributions at runtime
#2497 opened Nov 25, 2024 by Mutinifni
how to avoid oom when inference qwen2-vl 7B with batch=2?
#2496 opened Nov 25, 2024 by YSF-A
2
Qwen2-VL Batch Bug bug Something isn't working triaged Issue has been triaged by maintainers
#2495 opened Nov 25, 2024 by LugerW-A
2 of 4 tasks
1
[bug] forwardAsync assertion failed Generic Runtime triaged Issue has been triaged by maintainers
#2494 opened Nov 25, 2024 by akhoroshev
@nekorobov
3
Wrong output on Llama 3.2 1B, but 3B ok triaged Issue has been triaged by maintainers waiting for feedback
#2492 opened Nov 24, 2024 by lucasavila00
1
[Question] Running custom Encoder Decoder model question Further information is requested triaged Issue has been triaged by maintainers
#2491 opened Nov 24, 2024 by AvivSham
1
How to configure max_num_tokens and max_batch_size as runtime params? question Further information is requested Triton Backend
#2490 opened Nov 24, 2024 by JoJoLev
Issues with installing on Windows bug Something isn't working installation
#2489 opened Nov 23, 2024 by PyroGenesis
1 of 4 tasks
2
In streaming output mode, some Chinese characters are decoded as garbled characters
#2488 opened Nov 23, 2024 by HongfengDu
1
int4 not faster than fp16 and fp8 Performance Issue about performance number
#2487 opened Nov 22, 2024 by ShuaiShao93
4 tasks
3
Inconsistency with penaltyKernels.cu bug Something isn't working triaged Issue has been triaged by maintainers
#2486 opened Nov 22, 2024 by buddhapuneeth
2 of 4 tasks
@nekorobov
1
QwenVL build failed. bug Something isn't working triaged Issue has been triaged by maintainers
#2483 opened Nov 22, 2024 by Wonder-donbury
2 of 4 tasks
@sunnyqgg
2
Medusa performance degrades with batch size larger than 1 Performance Issue about performance number
#2482 opened Nov 22, 2024 by SoundProvider
2
How to install tensorrt-llm in python3.11? installation question Further information is requested
#2481 opened Nov 22, 2024 by janelu9
2
Can't uniquely locate model_spec module installation triaged Issue has been triaged by maintainers
#2480 opened Nov 21, 2024 by weizhi-wang
3
error: make -C docker release_build : Command 'git submodule update --init --recursive' returned non-zero exit status 128 installation triaged Issue has been triaged by maintainers
#2479 opened Nov 21, 2024 by xddun
1 of 4 tasks
3
[bug] Medusa example fails with vicuna 33B
#2478 opened Nov 21, 2024 by SoundProvider
2
trtllm-bench fail triaged Issue has been triaged by maintainers
#2476 opened Nov 21, 2024 by Wowoho
1
undefined reference to `__libc_single_threaded' bug Something isn't working installation triaged Issue has been triaged by maintainers
#2475 opened Nov 21, 2024 by hoangvictor
1 of 4 tasks
2
Determinism in TensorRT-LLM
#2474 opened Nov 20, 2024 by njaramish
ProTip! Updated in the last three days: updated:>2024-11-24.