Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: gradio integration #5008

Merged
merged 5 commits into from
Oct 11, 2024
Merged

feat: gradio integration #5008

merged 5 commits into from
Oct 11, 2024

Conversation

parano
Copy link
Member

@parano parano commented Oct 6, 2024

What does this PR address?

This PR simplifies how users can serve Gradio UI for their BentoML service. Previously user will need to manually do multiple settings and callbacks, e.g.:

import bentoml
import gradio as gr

def summarize_text(text: str) -> str:
    svc_instance = bentoml.get_current_service()
    return svc_instance.summarize([text])[0]

io = gr.Interface(
    fn=summarize_text,
    inputs=[gr.Textbox(lines=5, label="Enter Text", value=EXAMPLE_INPUT)],
    outputs=[gr.Textbox(label="Summary Text")],
    title="Summarization",
    description="Enter text to get summarized text."
)
gradio_app = gr.routes.App.create_app(io)

io.dev_mode = False
io.show_error = True
io.validate_queue_settings()

@bentoml.mount_asgi_app(gradio_app, path="/ui")
@bentoml.service(resources={"cpu": "4"})
class Summarization:

    def __init__(self) -> None:
        import torch
        from transformers import pipeline

        # gradio_app.get_blocks().queue()
        gradio_app.get_blocks().startup_events()
        # gradio_app.get_blocks().root_path="/ui"

        ...

With this PR, this can be simplified as:

def summarize_text(text: str) -> str:
    svc_instance = bentoml.get_current_service()
    return svc_instance.summarize([text])[0]

io = gr.Interface(
    fn=summarize_text,
    inputs=[gr.Textbox(lines=5, label="Enter Text", value=EXAMPLE_INPUT)],
    outputs=[gr.Textbox(label="Summary Text")],
    title="Summarization",
    description="Enter text to get summarized text."
)

@bentoml.service(resources={"cpu": "4"})
@bentoml.gradio.mount_gradio_app(io, path="/ui")
class Summarization:
      ...

Before submitting:

@parano parano requested a review from a team as a code owner October 6, 2024 10:13
@parano parano requested review from frostming and larme and removed request for a team and frostming October 6, 2024 10:13
Copy link
Contributor

@aarnphm aarnphm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably want frost opinion on this:

For any integration going forward, should it be under bentoml._internal or _bentoml_sdk?

I think it should be the second option?

src/_bentoml_sdk/service/factory.py Show resolved Hide resolved
src/bentoml/_internal/gradio.py Outdated Show resolved Hide resolved
examples/gradio/service.py Show resolved Hide resolved
src/bentoml/_internal/gradio.py Outdated Show resolved Hide resolved
@frostming frostming merged commit 68fc547 into bentoml:main Oct 11, 2024
50 of 51 checks passed
@Sherlock113 Sherlock113 mentioned this pull request Oct 23, 2024
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants