Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Misc] Set block size at initialization & Fix test_model_runner #4705

Merged
merged 7 commits into from
May 9, 2024

Conversation

WoosukKwon
Copy link
Collaborator

This PR sets the block size to ModelRunner at its initialization time, instead of setting it lazily. Currently, we don't have any reason to set this value lazily, since it is configured by the user argument.
Also, the PR fixes the abuse of ModelRunner in test_model_runner where basically ModelRunner was initialized with None arguments.

@WoosukKwon WoosukKwon requested a review from rkooo567 May 9, 2024 07:42
Copy link
Collaborator

@rkooo567 rkooo567 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to see this change!

@WoosukKwon WoosukKwon merged commit 0ee535b into main May 9, 2024
52 of 55 checks passed
@WoosukKwon WoosukKwon deleted the set-block-size branch May 9, 2024 16:05
Temirulan pushed a commit to Temirulan/vllm-whisper that referenced this pull request Sep 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants