-
Notifications
You must be signed in to change notification settings - Fork 710
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Updated documentation for Grammar Backend #2545
Updated documentation for Grammar Backend #2545
Conversation
@@ -223,6 +223,13 @@ | |||
"## Structured decoding (JSON, Regex)\n", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can define a JSON schema or regular expression to constrain the model's output, which depends on the grammar backend.
SGlang has two backends: outlines (default) and Xgrammar. Xgrammar enhances JSON decoding performance but does not support regular expressions. To use Xgrammar, add --grammar-backend xgrammar when launching the server:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated and verified the functionality with Xgrammar backend.
This is just to update the current document, so a tiny modification is good enough. However, for the constraint decoding tutorial, please truly launch the server to check its correctness. Also, what's the status of the tutorial? @shuaills |
I'm working on it. Should it be in Jupyter Notebook format? And where should I put the tutorial? |
It should be a Notebook. Put it under |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just tiny modification. Then we can merge it.
Motivation
Improve documentation clarity around grammar backends.
#2505
Modifications
Updated documentation to clarify Xgrammar's JSON support and regex limitation.
Checklist