Conflicting (kind of) information about bentoml serve

[ReubenTheDS]

BentoML Version: 1.0.11
Is it possible to run the command 'bentoml serve' in production with an imported Bento? The docs are kind of confusing:
The second para on https://docs.bentoml.org/en/latest/concepts/bento.html:
... Bento standardizes how to reproduce the required environment for running a bentoml.Service in production.
The same page goes on to show how a Bento can be built with 'bentoml build' and a 'bentofile.yaml'
Note the usage of the word 'production'

Now, the API reference for bentoml cli at https://docs.bentoml.org/en/latest/reference/cli.html#bentoml-serve says:
Start a HTTP BentoServer from a given 🍱

BENTO is the serving target, it can be the import as:

• the import path of a ‘bentoml.Service’ instance
• a tag to a Bento in local Bento store
• a folder containing a valid ‘bentofile.yaml’ build file with a ‘service’ field, which provides the import path of a ‘bentoml.Service’ instance
• a path to a built Bento (for internal & debug use only)

Note that the last option says 'internal & debug use only'

But, isn't a Bento built with 'bentoml build'?? Can we not export that Bento, then import it & serve it in production mode?

[dmsfabiano]

I agree, its super confusing. Also, why can't we start a service from a bentoml.Service object?!

[aarnphm]

You might be interested in using the bentoml.HTTPServer and bentoml.GrpcServer to start the server given a bento name or service.py:svc from the recent releases.

Wrt starting a service from bentoml.Service, one of the issue of creating bentoml.Service is that we are importing the service.py file into PYTHONPATH, in which we have ran into a lot of problems with import conflict with python import and user imports.

Unless there is a way we can structure a bentoml.Service without actual import the service.py, then starting a service from bentoml.Service is not feasible atm.

[parano]

Made a PR to support this scenario here: #3829

[aarnphm]

BentoML Version: 1.0.11 Is it possible to run the command 'bentoml serve' in production with an imported Bento?

Not sure if I understand this fully, but you can bentoml serve /path/to/bento , bentoml serve bento_name:tag

• the import path of a ‘bentoml.Service’ instance
• a tag to a Bento in local Bento store
• a folder containing a valid ‘bentofile.yaml’ build file with a ‘service’ field, which provides the import path of a ‘bentoml.Service’ instance
• a path to a built Bento (for internal & debug use only)

You can serve from a path, but it is a anti-pattern, hence the internal and debug use only. This is mainly for testing out the bento and make sure you package and bundle correct files.

Bento should be managed by the BentoML's bento store, and hence the recommended way is bentoml serve bento_name:tag. But you are free to use whatever you may find fit.

But, isn't a Bento built with 'bentoml build'?? Can we not export that Bento, then import it & serve it in production mode?

You can export the bento, import somewhere else and serve it

bentoml export bento:tag ./exported.bento

Then you can send that exported.bento to a second machine, it can do

bentoml import /path/to/exported.bento

bentoml serve bento:tag 

Hope this clarifies some of the documentation. We will probably need to clean and sync up our documentation.