Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Update weights from disk ends in runtime corruption for different size model #2322

Open
5 tasks done
zhaochenyang20 opened this issue Dec 2, 2024 · 1 comment · Fixed by #2323
Open
5 tasks done
Assignees

Comments

@zhaochenyang20
Copy link
Collaborator

Checklist

  • 1. I have searched related issues but cannot get the expected help.
  • 2. The bug has not been fixed in the latest version.
  • 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
  • 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
  • 5. Please use English, otherwise it will be closed.

Describe the bug

In our unit test test_update_weights_from_disk.py, we only test the update of the Engine with a wrong model name. But if you update llama 3.2 1B with llama 3.2 3B, the engine will be corrupted. This is found in our document native_api.ipynb. I now change our native_api.ipynb to be the same as our unit test. But I fix the error of updating weights from disk with a model of a different size.

Reproduction

Easy to reproduce.

Environment

Easy to say.

@zhaochenyang20
Copy link
Collaborator Author

This should not be closed since this bug is not fixed right now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant