Does the repo suport qwen-vl?

[tangyipeng100]

How to build qwen-vl using TensorRT-LLM?

[juney-nvidia]

@tangyipeng100

Hi, we do have an internal implementation supporting qwen-vl with TensorRT-LLM, it has not been published onto the github yet.

Let me sync with the engineers working on that and go back to you later.

June

[Yang-x-Zhao]

@juney-nvidia
Same request for me and thanks for your work!

I am also working on qwen-vl-chat. To my best understanding on TensorRT-LLM, the decoder session currently does not support embedding as an input, so I cannot modify the current qwen model to adapt to qwen-vl-chat model. I will be glad to use an official release of qwen-vl-chat.

[CN-COTER]

@tangyipeng100

Hi, we do have an internal implementation supporting qwen-vl with TensorRT-LLM, it has not been published onto the github yet.

Let me sync with the engineers working on that and go back to you later.

June

I am looking forward to deploy Qwen-vl-chat with trt-llm.

[77h2l]

same strong demands here .

[shikimoon]

Same request for me and thanks for your work!

I am also working on qwen-vl-chat. To my best understanding on TensorRT-LLM, the decoder session currently does not support embedding as an input, so I cannot modify the current qwen model to adapt to qwen-vl-chat model. I will be glad to use an official release of qwen-vl-chat.

the same to me

[Tlntin]

There might be what you're looking for here
link

[tangyipeng100]

There might be what you're looking for here link

Thanks!