使用vllm部署Chinese-LLaVA-Med的Chinese-LLaVA-Med-7B 失败

[lxb0425]

Reminder

• I have read the README and searched the existing issues.

System Info

基于llava1.5-7b-chat微调
CUDA_VISIBLE_DEVICES=0,1 API_PORT=7864 llamafactory-cli api /data/rich/project/LLaMA-Factory-main/examples/inference/vllm.yaml

vllm.yaml
model_name_or_path: /data/rich/model/Med-v1.0
template: vicuna
infer_backend: vllm
vllm_enforce_eager: true
这种部署可以文本对话 传图片报错

Reproduction

llamafactory-cli webchat --model_name_or_path /data/rich/model/Med-v1.0 --template vicuna --visual_inputs
这种方式界面是可以的
api的方式怎么实现

llamafactory-cli api config/llava1_5_full_sft_infer.yaml 后台报错 422
{
"model": "llava-v1.5",
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "请告诉我这张图像里有什么，中文回复我"
},
{
"type": "image_url",
"image_url": {
"url": "https://th.bing.com/th/id/R.6b5df1bfe0e4778a44dba0753cd169c8?rik=QRQIMqvjWRCO5Q&riu=http%3a%2f%2fpic39.nipic.com%2f20140321%2f8857347_232251363165_2.jpg&ehk=7oAaMo6LCHJc%2bqpQ0IPvcH7v69jGRQhb2vDz%2fOd5720%3d&risl=&pid=ImgRaw&r=0"
}
}
]
}
],
"temperature": 0.7,
"top_p": 1,
"logprobs": false,
"top_logprobs": 0,
"n": 1,
"max_tokens": null,
"stop": null,
"stream": false,
"presence_penalty": 0,
"frequency_penalty": 0,
"user": "string",
"repetition_penalty": 1,
"session_id": -1,
"ignore_eos": false,
"skip_special_tokens": true,
"top_k": 40
}### Expected behavior

No response

Others

No response

[injet-zhou]

The inference API currently does not support image input.
api模式下目前还不支持图片输入。

[lxb0425]

llava官方部署没有微调的可以正常输入图像，部署使用llama factory微调后的版本就是失败 llama factory的api部署又不支持图形输入 就卡了bug

[hiyouga]

api 已经支持图像

[lxb0425]

入参转成base64 直接超token了

[hiyouga]

@lxb0425 修复了