Add Multimodal LLM Finetuning

[BUAADreamer]

What does this PR do?

Add finetuning Multimodal-LLM especially for LLaVA by leveraging AutoModelForVision2Seq and AutoProcessortransformers

This PR is working in progress, needs improvement in the future, e.g. other MLLM.

For more usage, you can refer to MLLM-Finetuning-Demo

Support Models

• LLaVA-1.5

Make your own Instruct Dataset

Just organize the content like the data/mllm_demo.json.

Finetuning

See examples at examples/lora_single_gpu/llava1_5_lora_sft.yaml

Before submitting

• Did you read the contributor guideline?

[hiyouga]

LGTM! Thanks for your contribs!

[whyiug]

Hi @BUAADreamer, thanks for your work. Can you explain how this HF version differs from the original(i mean https://github.com/haotian-liu/LLaVA) during training, and does the HF version trains the mm_projector layers?
thanks a lot.

[BUAADreamer]

This HFversion is nearly the same as the origin, it is transferred by the official researchers of Huggingface and Haotian Liu.
Our current sft of MLLM is the same as the ft stage in LLaVA paper, only fine-tune the mm_proj and the LM.
You could refer to this Zhihu blog to learn more about fine-tuning MLLM
And you could refer to this fine-tuned paligemma by @hiyouga for a successful example

[BUAADreamer]

Besides, if you want to pre-train like LLaVA, you can refer to #3835 to only fine-tune the mm_proj

[whyiug]

Thx, those really help me a lot.