Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

推理时llava.eval.model_vqa中的metadata是否忘记改为category? #17

Open
wxxu-cnic opened this issue Oct 16, 2024 · 2 comments

Comments

@wxxu-cnic
Copy link

ans_file.write(json.dumps({"question_id": idx,
"prompt": cur_prompt,
"text": outputs,
"answer_id": ans_id,
"model_id": model_name,
"metadata": {}}) + "\n")
然而输出格式要求为
{ 'question_id': 'TABMWP_8', # item_id
'prompt': 'Problem: \nHannah baked cookies each day ...', # input_prompt
'text': 'Find the numbers in the table.\n\nSaturday: ...', # model_output
'answer_id': 'jELcxSPcXHBj3xvHfm5r8T', # answer_id
'model_id': 'table-llava-7b', # model_id
'category': 'TABMWP_for_TQA'
} # item category

@wxxu-cnic
Copy link
Author

ans_file.write(json.dumps({"question_id": idx, "prompt": cur_prompt, "text": outputs, "answer_id": ans_id, "model_id": model_name, "metadata": {}}) + "\n") 然而输出格式要求为 { 'question_id': 'TABMWP_8', # item_id 'prompt': 'Problem: \nHannah baked cookies each day ...', # input_prompt 'text': 'Find the numbers in the table.\n\nSaturday: ...', # model_output 'answer_id': 'jELcxSPcXHBj3xvHfm5r8T', # answer_id 'model_id': 'table-llava-7b', # model_id 'category': 'TABMWP_for_TQA' } # item category

我按照给定脚本运行无法得到category

@SpursGoZmy
Copy link
Owner

感谢同学指正,可能是我上传的llava代码版本太新了,和老版本有改动,然后sh脚本对应的是老脚本,我后面修正一下。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants