New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

多选题没有sort #3

Open

wjfwzzc opened this issue May 19, 2023 · 1 comment

wjfwzzc commented May 19, 2023

如：

MMCU/test_results/gpt3.5turbo_test_results_0512/zero_shot/寄生虫学100题

Line 97 in a506f23

B|||ACBD|||False

所示，模型的预测可能选项正确但顺序不对，答案中也有选项顺序不对的情况。所以评测脚本是不是应该在判断选项正确之前sort一下预测和答案？

Owner

Felixgithub2017 commented May 23, 2023

感谢反馈，我们尽快修正数据集以及脚本

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment