You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
2. Please use English, otherwise it will be closed.
Motivation
The recent addition of EAGLE speculative decoding in here is powerful. Thank you for creating and maintaining such a useful tool! The existing codebase gives insufficient examples of how it can be used (e.g for Llama3 models, for example) together with docker compose. It would be great if another file like https://github.com/sgl-project/sglang/blob/main/docker/compose.yaml can be added to illustrate how the feature can be used in docker environments. Thanks for looking into this issue!
The text was updated successfully, but these errors were encountered:
daviddl9
changed the title
[Feature] docs: Improve documentation on how to use EAGLE speculative docoding
[Feature] docs: Improve documentation on how to use EAGLE speculative docoding with docker-compose.ymlJan 23, 2025
@simveit Hey Simon. Thanks so much. We can work on accuracy tests first. I will let Jin @jhinpan to take this~
zhaochenyang20
changed the title
[Feature] docs: Improve documentation on how to use EAGLE speculative docoding with docker-compose.yml
[Feature] docs: Improve documentation on how to use EAGLE speculative docoding
Jan 24, 2025
Checklist
Motivation
The recent addition of EAGLE speculative decoding in here is powerful. Thank you for creating and maintaining such a useful tool! The existing codebase gives insufficient examples of how it can be used (e.g for Llama3 models, for example) together with
docker compose
. It would be great if another file like https://github.com/sgl-project/sglang/blob/main/docker/compose.yaml can be added to illustrate how the feature can be used in docker environments. Thanks for looking into this issue!The text was updated successfully, but these errors were encountered: