About the inference

[srymaker]

Thanks for your great work!
I want to know, when I want to do style transfer task, do I need to input a reference picture, a style word corresponding to this reference picture and a target prompt to the model? Just like This triplet <reference img,reference style word,target prompt>

[Tianhao-Qi]

No, you only need a pair <reference image, target prompt>.

[srymaker]

Thank you for your answer. So what are the inputs and target during the training?

[Tianhao-Qi]

There are three kinds of training pairs:

1. reference and target images are with the same style, but distinct subjects (STRE);
2. reference and target images are with the same subject, but distinct styles (SERE);
3. reference and target images are the same (Reconstruction).
   You can refer to Sec 3.2 in our paper.

[srymaker]

Thank you,but in the paper,the qforme’s input should have the text {content} or {style},what is it

[Tianhao-Qi]

The text input of Q-former is the word "content" or "style".

[srymaker]

Oh,i see,thanks for your patience

[SkylerZheng]

Hi @Tianhao-Qi , does the current released code support this "Stylized Reference Object Generation" function? Basically I want to convert my given image to a different style by providing the text only, the given image is the source image rather than the style image.

[Tianhao-Qi]

You can refer to this script. Besides, if you want to keep the structure of the source image as well, you'll need to use the controlnet.

[LiamLiu62]

There are three kinds of training pairs:

1. reference and target images are with the same style, but distinct subjects (STRE);
2. reference and target images are with the same subject, but distinct styles (SERE);
3. reference and target images are the same (Reconstruction).
   You can refer to Sec 3.2 in our paper.

In Dataset part, for "style", your paper says use the same prompts to generate the reference and target image. So, i think they should be in the same subject?