Doubts about the evaluation results

[Yanllan]

First of all, congratulations on completing such a work, but I am confused about the results in the technical report.
The reported results are referred to as "zero-shot evaluation of RadFM", however, the vqa-rad dataset exists in your training dataset, isn't it somewhat paradoxical that? Or did you do something with the dataset or did I misinterpret the paper?

[chaoyi-wu]

Thanks for your question. Here our “zero-shot” setting is more referred as a prompting method, distinguished with few-shot, CoT and so on, following FLAN https://arxiv.org/pdf/2301.13688. In other word, "zero-shot" here is not directly linked with any domain shift.

We talk about the transferring ability on unseen tasks in section "Generalization to Unseen Classes in PadChest" which may be more aligned with your understanding for "zero-shot".