Skip to content

Commit

Permalink
[Doc] Update mllama example based on official doc (#11567)
Browse files Browse the repository at this point in the history
Signed-off-by: Chen Zhang <[email protected]>
  • Loading branch information
heheda12345 authored Dec 28, 2024
1 parent ac79799 commit a607312
Showing 1 changed file with 14 additions and 1 deletion.
15 changes: 14 additions & 1 deletion examples/offline_inference_vision_language.py
Original file line number Diff line number Diff line change
Expand Up @@ -308,7 +308,20 @@ def run_mllama(question: str, modality: str):
disable_mm_preprocessor_cache=args.disable_mm_preprocessor_cache,
)

prompt = f"<|image|><|begin_of_text|>{question}"
tokenizer = AutoTokenizer.from_pretrained(model_name)
messages = [{
"role":
"user",
"content": [{
"type": "image"
}, {
"type": "text",
"text": f"{question}"
}]
}]
prompt = tokenizer.apply_chat_template(messages,
add_generation_prompt=True,
tokenize=False)
stop_token_ids = None
return llm, prompt, stop_token_ids

Expand Down

0 comments on commit a607312

Please sign in to comment.