I'm having trouble understanding how to run a model graded eval #1341
Unanswered
mayalinetsky-kryon
asked this question in
Q&A
Replies: 2 comments
-
Hey, I've recently got 2 pull requests approved on this, check out gpt4facts and A is B, B is A pull requests. Something that also helps is to checkout merged pull requests that used modelgraded. There is also some documentation on them and I suggest reading the files on modelgraded to see what it is doing and how it works. |
Beta Was this translation helpful? Give feedback.
0 replies
-
I also have hard time understanding how to use model graded eval, could someone please give a concrete example (like a command)? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hey,
I've read the documentations about how to run an eval, but I could not figure out how to run the ones under evals/registry/modelgraded (and specifically I want closedqa.yaml).
I'm interested in running a modelgraded eval on gpt-3.5-turbo with a custom system message.
How do I do that?
Where does the system message come in?
Beta Was this translation helpful? Give feedback.
All reactions