Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nit: Remove busy waiting on scheduler #2382

Merged
merged 4 commits into from
Dec 8, 2024

Conversation

rkooo567
Copy link
Contributor

@rkooo567 rkooo567 commented Dec 7, 2024

Motivation

This PR includes 3 changes.

  • contributor guide has no instruction regarding build. I found it is in install page, and I added a link there.
  • Use setproctitle to change the process title for sglang. (it is contraversial, and we need to introduce a new dependency).
  • Scheduler has busy waiting on recv_pyobject even when there are no requests which makes scheduler have 100% cpu usage even when the engine is idle. I make it use poll API when there's no batch to run (we need busy waiting when there are batches because we need to keep running batches).

Modifications

Checklist

  • Format your code according to the Contributor Guide.
  • Add unit tests as outlined in the Contributor Guide.
  • Update documentation as needed, including docstrings or example tutorials.

@rkooo567
Copy link
Contributor Author

rkooo567 commented Dec 7, 2024

I just made a PR to start contribution. Not sure if change makes sense to sglang perspective...

Copy link
Collaborator

@ByronHsu ByronHsu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

setproctitle looks pretty useful!

@merrymercy merrymercy merged commit 1f09e84 into sgl-project:main Dec 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants