Skip to content

Commit

Permalink
update clis.rst
Browse files Browse the repository at this point in the history
[Issue](flyteorg/flyte#2640)
Signed-off-by: SmritiSatyanV [email protected]
  • Loading branch information
SmritiSatyanV committed Aug 4, 2022
1 parent 96c0b3c commit af9fef3
Showing 1 changed file with 46 additions and 6 deletions.
52 changes: 46 additions & 6 deletions docs/source/design/clis.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,54 @@ The client code is located in ``flytekit/clients`` and there are two.
Command Line Interfaces
***********************

Flyte CLI
Flytectl
===========
``flyte-cli`` is the general CLI that can be used to talk to the Flyte control plane (Flyte Admin). It ships with Flytekit as part of the Pypi package. Think of this as the ``kubectl`` for Flyte. In fact, we're working on ``flytectl`` which is under active development the completion of which will deprecate this CLI.

Think of this CLI as a network-aware (i.e. can talk to Admin) but not code-aware (doesn't need to have user code checked out) CLI. In the registration flow, this CLI is responsible for shipping the compiled Protobuf files off to Flyte Admin.
``flytectl`` is the general CLI that can be used to communicate with the Flyte control plane (FlyteAdmin). It ships with flytekit as part of the Pypi package. Think of this as the ``kubectl`` for Flyte.
Think of this CLI as a network-aware (can talk to Admin) but not code-aware (doesn't need to have user code checked out) CLI. In the registration flow, this CLI is responsible for shipping the compiled Protobuf files off to Flyte Admin.

Pyflyte
=========
Unlike Flyte CLI, think of this CLI as code-aware, but not network-aware (the latter is not entirely true, but it's helpful to think of it that way).
Unlike flytectl, think of this CLI as code-aware, but not network-aware (the latter is not entirely true, but it's helpful to think of it that way).

This CLI is responsible for the serialization (compilation) step in the registration flow. It will parse through the user code, looking for tasks, workflows and launch plans, and compile them to protobuf files.

What is ``pyflyte register``?
=============================
``pyflyte register`` is a command that registers all the workflows present in the repository/directory using fast-registration. It is equivalent to using two commands (``pyflyte package`` and ``flytectl register``) to perform the same operation (registration).
The ``pyflyte package`` command parses and compiles the user’s Python code into Flyte protobuf objects.
The ``flytectl register`` command ships the protobuf objects over the network to Flyte control plane, wherein ``flytectl`` also allows adding attributes such as IAM role, K8s service account, and so on.

Why should you use ``pyflyte register``?
========================================
The ``pyflyte register`` command bridges the gap between ``pyflyte package`` + ``flytectl register``and ``pyflyte run`` commands. It offers the functionality of the ``pyflyte package`` (with smarter naming semantics and combines the network call into one step), by registering all the workflows in the repository, and the convenience of the ``pyflyte run``, by launching the execution of a single workflow.

.. note ::
You can’t use ``pyflyte register`` if you are unaware of the run-time options yet(IAM role, service account, and so on).
What is ``pyflyte run``?
========================
The ``pyflyte run`` command is a light-weight, convenience command that combines packaging, registering and launching a workflow in a single command.
It is not a fully featured production scale mode of operation, because it
is designed to be a quick and easy iteration tool to get started with Flyte or test small self-contained scripts. The caveat here is it operates on a single file, and this file will have to contain all the required Flyte entities. Let’s take an example so that you can understand it better.
Suppose you execute a script that defines 10 tasks and a workflow that calls only 2 out of the 10 tasks, the remaining 8 tasks don’t get registered at that point.
It is considered fast registration because when a script is executed using ``pyflyte run``, the script is zipped up and sent to FlyteAdmin. When the task is executed in the backend, this zipped file is extracted and used.

Usage
=====

.. prompt:: bash $

pyflyte --config ~/.flyte/dev-uniondemo.yaml register --image ghcr.io/flyteorg/flytecookbook:core-latest --image trainer=ghcr.io/flyteorg/flytecookbook:core-latest --image predictor=ghcr.io/flyteorg/flytecookbook:core-latest --raw-data-prefix s3://development-service-flyte/reltsts flyte_basics

.. note ::
Neither ``pyflyte register`` nor ``pyflyte run`` commands work on Python namespace packages since both the tools traverse the filesystem to find the first folder that doesn't have an __init__.py file, which is interpreted as the root of the project. Both the commands use this root as the basis to name the Flyte entities.
In a broad way, ``pyflyte register`` is equivalent to ``pyflyte run`` minus launching workflow with the exception that ``pyflyte run`` can only register a single workflow whereas ``pyflyte register`` can register all workflows in a repository.

Should I use ``pyflyte run`` or ``pyflyte package`` + ``flytectl register``?
============================================================================
``pyflyte run`` is convenient when you are starting out with Flyte, and testing small scripts.
``pyflyte package`` + ``flytectl register`` is convenient when you want to zip your files (that constitutes all your tasks and workflows) into a single portable file and pass it on to another developer to register it.

This CLI is what is responsible for the serialization (compilation) step in the registration flow. It will parse through user code, looking for tasks, workflows and launch plans, and compile them down to Protobuf files.

0 comments on commit af9fef3

Please sign in to comment.