Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[router] Improve cleanup logic #2411

Merged
merged 4 commits into from
Dec 8, 2024
Merged

[router] Improve cleanup logic #2411

merged 4 commits into from
Dec 8, 2024

Conversation

ByronHsu
Copy link
Collaborator

@ByronHsu ByronHsu commented Dec 8, 2024

The current sglang_router.sglang_server has overcomplicated logic of process cleanup

  1. Replace force kill with terminate because sglang server handles graceful termination internally. It is safer and more predictable too.
  2. Remove exception catching because it is already handled by python internal.

The current test_launch_server has a bug

  1. We shutdown the router server per class, but it should be per test case. The test passed just by luck because python ignores the port occupation error.

Depends on #2410

@ByronHsu ByronHsu requested a review from Ying1123 as a code owner December 8, 2024 22:05
@ByronHsu ByronHsu force-pushed the byhsu/improve-cleanup branch from fdd47e8 to 1f63d63 Compare December 8, 2024 22:17
@ByronHsu ByronHsu force-pushed the byhsu/improve-cleanup branch from 1f63d63 to 5554a30 Compare December 8, 2024 22:17
@ByronHsu ByronHsu merged commit a1e697b into main Dec 8, 2024
6 checks passed
@ByronHsu ByronHsu deleted the byhsu/improve-cleanup branch December 8, 2024 23:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant