Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow starting server with bentoml.Service instance #3829

Merged

Conversation

parano
Copy link
Member

@parano parano commented May 5, 2023

What does this PR address?

This PR made the following use case possible, this is especially helpful in a project development environment where no Bento was built yet.

import bentoml

# import the Service defined in `/clip_api_service/service.py` file
from clip_api_service.service import svc 

# start a server:
server = bentoml.HTTPServer(svc)
server.start(blocking=False)
client = server.get_client()
client.rank(..)

Similarly, users can also put a launch script within a service definition file and run it as a regular Python program:

# service.py
svc = bentoml.Service(...)

if __name__ == "__main__":
    server = bentoml.HTTPServer(svc)
    server.start(blocking=True)
python service.py

See discussion here: #3560 (comment)

Fixes #(issue)

Before submitting:

@parano parano requested a review from a team as a code owner May 5, 2023 01:12
@parano parano requested review from ssheng and removed request for a team May 5, 2023 01:12
@codecov
Copy link

codecov bot commented May 5, 2023

Codecov Report

Merging #3829 (3811528) into main (f3ef39a) will not change coverage.
The diff coverage is 0.00%.

Impacted file tree graph

@@          Coverage Diff          @@
##            main   #3829   +/-   ##
=====================================
  Coverage   0.00%   0.00%           
=====================================
  Files        154     154           
  Lines      12682   12736   +54     
=====================================
- Misses     12682   12736   +54     
Impacted Files Coverage Δ
src/bentoml/__init__.py 0.00% <ø> (ø)
src/bentoml/_internal/bento/bento.py 0.00% <0.00%> (ø)
src/bentoml/_internal/service/loader.py 0.00% <0.00%> (ø)
src/bentoml/_internal/service/service.py 0.00% <0.00%> (ø)
src/bentoml/server.py 0.00% <0.00%> (ø)

... and 1 file with indirect coverage changes

@parano parano added the pr/merge-hold Requires further discussions before a pull request can be merged label May 5, 2023
@parano parano removed the pr/merge-hold Requires further discussions before a pull request can be merged label May 5, 2023
src/bentoml/server.py Outdated Show resolved Hide resolved
ssheng
ssheng previously approved these changes May 5, 2023
@aarnphm aarnphm merged commit e2edce6 into bentoml:main May 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants