Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Support reward model LxzGordon/URM-LLaMa-3.1-8B #1525

Merged
merged 2 commits into from
Sep 28, 2024
Merged

Conversation

Ying1123
Copy link
Member

@Ying1123 Ying1123 commented Sep 27, 2024

This PR adds support for the reward model LxzGordon/URM-LLaMa-3.1-8B. It currently supports SGLang native REST API of http://{host}:{port}/judge.

Example usage:

# launch server
python -m sglang.launch_server --model LxzGordon/URM-LLaMa-3.1-8B --is-embedding
# send requests
import json

import requests

url = "http://127.0.0.1:30000"

PROMPT = (
    "What is the range of the numeric output of a sigmoid node in a neural network?"
)
RESPONSE1 = "The output of a sigmoid node is bounded between -1 and 1."
RESPONSE2 = "The output of a sigmoid node is bounded between 0 and 1."

json_data = {
    "conv": [
        [
            {"role": "user", "content": PROMPT},
            {"role": "assistant", "content": RESPONSE1},
        ],
        [
            {"role": "user", "content": PROMPT},
            {"role": "assistant", "content": RESPONSE2},
        ],
    ],
}
response = requests.post(
    url + "/judge",
    json=json_data,
).json()

print(response)
print("scores:", [x["embedding"] for x in response])

@Ying1123 Ying1123 force-pushed the ying-reward branch 4 times, most recently from c7adc06 to 9b9eb48 Compare September 27, 2024 22:09
@Ying1123 Ying1123 merged commit 9aa6553 into main Sep 28, 2024
11 checks passed
@Ying1123 Ying1123 deleted the ying-reward branch September 28, 2024 06:32
@Ying1123 Ying1123 mentioned this pull request Sep 28, 2024
37 tasks
@merrymercy merrymercy mentioned this pull request Oct 19, 2024
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant