Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New Task Request: InflectionAI's Physics GRE #1554

Open
RylanSchaeffer opened this issue Mar 10, 2024 · 1 comment · May be fixed by #1655
Open

New Task Request: InflectionAI's Physics GRE #1554

RylanSchaeffer opened this issue Mar 10, 2024 · 1 comment · May be fixed by #1655
Labels
feature request A feature that isn't implemented yet. good first issue Good for newcomers help wanted Contributors and extra help welcome.

Comments

@RylanSchaeffer
Copy link
Contributor

There is a new dataset of Physics GRE exams constructed by Inflection AI: https://github.com/InflectionAI/Inflection-Benchmarks?tab=readme-ov-file#physics-gre

Adding this task would be a nice addition :)
@haileyschoelkopf haileyschoelkopf added help wanted Contributors and extra help welcome. feature request A feature that isn't implemented yet. good first issue Good for newcomers labels Mar 18, 2024
@ShayekhBinIslam
Copy link

@haileyschoelkopf I am interested in contributing here by adding this task if possible.

@ShayekhBinIslam ShayekhBinIslam linked a pull request Apr 1, 2024 that will close this issue
Open
@haileyschoelkopf haileyschoelkopf moved this from Ready to In progress in LM-Eval Support and Development Apr 1, 2024
@lintangsutawika lintangsutawika linked a pull request Apr 9, 2024 that will close this issue
Physics GRE task added #1655
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature request A feature that isn't implemented yet. good first issue Good for newcomers help wanted Contributors and extra help welcome.
Projects
LM-Eval Support and Development
Status: In progress
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Physics GRE task added ShayekhBinIslam/lm-evaluation-harness
3 participants
@RylanSchaeffer @ShayekhBinIslam @haileyschoelkopf