Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] humaneval_postprocess历史无效的单测清理或case更新 #1847

Open
1 task done
anfreshman opened this issue Jan 24, 2025 · 0 comments
Open
1 task done
Assignees

Comments

@anfreshman
Copy link

Describe the feature

【问题背景】
这边在本地拉取和构建opencompass的过程中,发现test_humaneval.py文件对应的单测无法正常运行,经过检查后,发现原单测所测试的humaneval_postprocess函数已经在 2024.06.28, commit id为 a32f21a3的一次提交中被迭代掉了,取而代之的是humaneval_postprocess_v2、humaneval_postprocess_v3

【希望确认的点】
humaneval_postprocess_v2函数 与 humaneval_postprocess_v3函数的逻辑看起来相当简洁,核心是正则表达式是否符合预期,我不确定项目团队是否有计划为这两个函数添加新的单测,如果没有的话或许可以直接删除test_humaneval,如果有的话可能需要为这两个函数涉及一套新的验证case
如果有我可以贡献力量的地方我会非常乐意创建PR

Will you implement it?

  • I would like to implement this feature and create a PR!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants