update clis.rst #1124
Merged
update clis.rst #1124
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[Issue](flyteorg/flyte#2640) Signed-off-by: SmritiSatyanV [email protected]
SmritiSatyanV
requested review from
wild-endeavor,
kumare3,
eapolinario and
pingsutw
as code owners
Codecov Report
@@ Coverage Diff @@
## master #1124 +/- ##
==========================================
+ Coverage 68.23% 68.26% +0.03%
==========================================
Files 287 287
Lines 25786 25829 +43
Branches 2882 2885 +3
==========================================
+ Hits 17595 17633 +38
- Misses 7713 7719 +6
+ Partials 478 477 -1
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
docs/source/design/clis.rst
Outdated
|
|
||
| Pyflyte | ||
| ========= | ||
| Unlike Flyte CLI, think of this CLI as code-aware, but not network-aware (the latter is not entirely true, but it's helpful to think of it that way). | ||
| Unlike flytectl, think of this CLI as code-aware, but not network-aware (the latter is not entirely true, but it's helpful to think of it that way). |
There was a problem hiding this comment.
the "not network-aware" part is no longer even remotely true. Let's remove that bit.
docs/source/design/clis.rst
Outdated
|
|
||
| You can’t use ``pyflyte register`` if you are unaware of the run-time options yet(IAM role, service account, and so on). | ||
|
|
||
| What is ``pyflyte run``? |
There was a problem hiding this comment.
can we move this run section above register?
| The ``pyflyte package`` command parses and compiles the user’s Python code into Flyte protobuf objects. | ||
| The ``flytectl register`` command ships the protobuf objects over the network to Flyte control plane, wherein ``flytectl`` also allows adding attributes such as IAM role, K8s service account, and so on. | ||
|
|
||
| Why should you use ``pyflyte register``? |
There was a problem hiding this comment.
also add that pyflyte register is only fast register for now. that is, it will always bundle up and upload the code, there is no way around that. We're going to add a non-fast version shortly.
docs/source/design/clis.rst
Outdated
|
|
||
| In a broad way, ``pyflyte register`` is equivalent to ``pyflyte run`` minus launching workflow with the exception that ``pyflyte run`` can only register a single workflow whereas ``pyflyte register`` can register all workflows in a repository. | ||
|
|
||
| Should I use ``pyflyte run`` or ``pyflyte package`` + ``flytectl register``? |
There was a problem hiding this comment.
Can we also differentiate between "pyflyte package + flytectl register" and just "pyflyte register"?
To me there is a difference. The first option will produce a "portable" package of your flyte entities. that is you can take that .tgz file, and register it with ten different flyte backends. you can email it to your friend at a different company and ask them to register it with their admin. you can also save the asset, keep it around for audit, etc.
pyflyte register works well for single admin use-cases, and the compiled objects are lost afterwards, so you can't as easily introspect. In a production Flyte setting, I think each has their place. If I were running something in CI, I would opt for the first route. If i were a user iterating locally, I'd probably opt for the second way. @kumare3 thoughts?
docs/source/design/clis.rst
Outdated
|
|
||
| Why should you use ``pyflyte register``? | ||
| ======================================== | ||
| The ``pyflyte register`` command bridges the gap between ``pyflyte package`` + ``flytectl register``and ``pyflyte run`` commands. It offers the functionality of the ``pyflyte package`` (with smarter naming semantics and combines the network call into one step), by registering all the workflows in the repository, and the convenience of the ``pyflyte run``, by launching the execution of a single workflow. |
There was a problem hiding this comment.
minor correction here - pyflyte register will not launch any executions. it'll just register them.
Changes based on comments Signed-off-by: SmritiSatyanV [email protected]
Fixed end-of-file error Signed-off-by: SmritiSatyanV [email protected]
docs/source/design/clis.rst
Outdated
| Flytectl | ||
| ========= | ||
|
|
||
| ``flytectl`` is the general CLI to communicate with the Flyte control plane (FlyteAdmin). It ships with flytekit as part of the Pypi package. Think of this as the ``kubectl`` for Flyte. |
There was a problem hiding this comment.
missed this the first time. flytectl does not ship with flytekit. it needs to be installed separately. Can we link to flytectl docs - https://docs.flyte.org/projects/flytectl/en/latest/
docs/source/design/clis.rst
Outdated
|
|
||
| ``pyflyte register`` is a command that registers all the workflows present in the repository/directory using fast-registration. It is equivalent to using two commands (``pyflyte package`` and ``flytectl register``) to perform the same operation (registration). | ||
|
|
||
| The ``pyflyte package`` command parses and compiles the user’s Python code into Flyte protobuf objects. These compiled objects are not stored, which makes it difficult to introspect the objects. |
There was a problem hiding this comment.
no they are stored, they're available locally after you run pyflyte package. In pyflyte register, they are not stored.
docs/source/design/clis.rst
Outdated
|
|
||
| The ``pyflyte package`` command parses and compiles the user’s Python code into Flyte protobuf objects. These compiled objects are not stored, which makes it difficult to introspect the objects. | ||
|
|
||
| The ``flytectl register`` command ships the protobuf objects over the network to the Flyte control plane, wherein ``flytectl`` also allows adding attributes such as IAM role, K8s service account, and so on. |
There was a problem hiding this comment.
Suggested change
| The ``flytectl register`` command ships the protobuf objects over the network to the Flyte control plane, wherein ``flytectl`` also allows adding attributes such as IAM role, K8s service account, and so on. | |
| The ``flytectl register`` command then ships the protobuf objects over the network to the Flyte control plane. In the process, ``flytectl`` also allows the user to set run-time attributes such as IAM role, K8s service account, etc. |
docs/source/design/clis.rst
Outdated
|
|
||
| It is considered fast registration because when a script is executed using ``pyflyte run``, the script is bundled up and uploaded to FlyteAdmin. When the task is executed in the backend, this zipped file is extracted and used. | ||
|
|
||
| ``pyflyte run`` produces a **portable** package (a .tgz file) of Flyte entities, which helps introspect them at a later time too. You can register this package with multiple Flyte backends. You can save this package, use it for audit, register with a different admin, and so on. |
There was a problem hiding this comment.
No pyflyte run does not do this. this is pyflyte package.
docs/source/design/clis.rst
Outdated
|
|
||
| .. prompt:: bash $ | ||
|
|
||
| pyflyte --config ~/.flyte/dev-uniondemo.yaml register --image ghcr.io/flyteorg/flytecookbook:core-latest --image trainer=ghcr.io/flyteorg/flytecookbook:core-latest --image predictor=ghcr.io/flyteorg/flytecookbook:core-latest --raw-data-prefix s3://development-service-flyte/reltsts flyte_basics |
There was a problem hiding this comment.
we should not use dev-uniondemo.yaml
Suggested change
| pyflyte --config ~/.flyte/dev-uniondemo.yaml register --image ghcr.io/flyteorg/flytecookbook:core-latest --image trainer=ghcr.io/flyteorg/flytecookbook:core-latest --image predictor=ghcr.io/flyteorg/flytecookbook:core-latest --raw-data-prefix s3://development-service-flyte/reltsts flyte_basics | |
| pyflyte register --image ghcr.io/flyteorg/flytecookbook:core-latest --image trainer=ghcr.io/flyteorg/flytecookbook:core-latest --image predictor=ghcr.io/flyteorg/flytecookbook:core-latest --raw-data-prefix s3://development-service-flyte/reltsts flyte_basics |
docs/source/design/clis.rst
Outdated
| Why should you use ``pyflyte register``? | ||
| ======================================== | ||
|
|
||
| The ``pyflyte register`` command bridges the gap between ``pyflyte package`` + ``flytectl register``and ``pyflyte run`` commands. It offers the functionality of the ``pyflyte package`` (with smarter naming semantics and combines the network call into one step), by only registering all the workflows in the repository. |
There was a problem hiding this comment.
Suggested change
| The ``pyflyte register`` command bridges the gap between ``pyflyte package`` + ``flytectl register``and ``pyflyte run`` commands. It offers the functionality of the ``pyflyte package`` (with smarter naming semantics and combines the network call into one step), by only registering all the workflows in the repository. | |
| The ``pyflyte register`` command bridges the gap between ``pyflyte package`` + ``flytectl register`` and ``pyflyte run`` commands. It offers the functionality of the ``pyflyte package`` (with smarter naming semantics and combines the network call into one step). |
docs/source/design/clis.rst
Outdated
|
|
||
| .. note :: | ||
|
|
||
| You can’t use ``pyflyte register`` if you are unaware of the run-time options yet(IAM role, service account, and so on). |
There was a problem hiding this comment.
Suggested change
| You can’t use ``pyflyte register`` if you are unaware of the run-time options yet(IAM role, service account, and so on). | |
| You can’t use ``pyflyte register`` if you are unaware of the run-time options yet (IAM role, service account, and so on). |
docs/source/design/clis.rst
Outdated
|
|
||
| This CLI is what is responsible for the serialization (compilation) step in the registration flow. It will parse through user code, looking for tasks, workflows and launch plans, and compile them down to Protobuf files. | ||
| ``pyflyte package`` + ``flytectl register`` is useful for single admin use-cases where you want to zip your files (that constitute all your tasks and workflows) into a single portable file and pass it on to another developer/location to register it or run something in CI. |
There was a problem hiding this comment.
Not sure what is meant by single admin use-case here. that comment is for pyflyte register, not package. making a portable package implies multiple admins.
Fixed wrong statements Signed-off-by: SmritiSatyanV [email protected]
pingsutw
previously approved these changes
Aug 12, 2022
docs/source/design/clis.rst
Outdated
| Unlike Flyte CLI, think of this CLI as code-aware, but not network-aware (the latter is not entirely true, but it's helpful to think of it that way). | ||
| ======== | ||
|
|
||
| Unlike ``flytectl``, think of this CLI as code-aware, which is responsible for the serialization (compilation) step in the registration flow. It will parse through the user code, looking for tasks, workflows, and launch plans, and compile them to protobuf files. |
There was a problem hiding this comment.
Suggested change
| Unlike ``flytectl``, think of this CLI as code-aware, which is responsible for the serialization (compilation) step in the registration flow. It will parse through the user code, looking for tasks, workflows, and launch plans, and compile them to protobuf files. | |
| Unlike ``flytectl``, think of this CLI as code-aware, which is responsible for the serialization (compilation) step in the registration flow. It will parse through the user code, looking for tasks, workflows, and launch plans, and compile them to `protobuf files <https://github.com/flyteorg/flyteidl/blob/0b20c5c99f9e964370d4f4ca663990ed56a14c7c/protos/flyteidl/core/workflow_closure.proto#L11-L18>`__. |
docs/source/design/clis.rst
Outdated
| What is the difference between ``pyflyte package + flytectl register`` and ``pyflyte register``? | ||
| ============================================================================================== | ||
|
|
||
| ``pyflyte package + flytectl register`` works well with multiple admins since it produces a portable package. You can also use it to run scripts in CI. |
There was a problem hiding this comment.
should we use FlyteAdmins instead of admins here?
docs/source/design/clis.rst
Outdated
|
|
||
| ``pyflyte package + flytectl register`` works well with multiple admins since it produces a portable package. You can also use it to run scripts in CI. | ||
|
|
||
| ``pyflyte register`` works well in single admin use-cases and cases where you are iterating locally. |
docs/source/design/clis.rst
Outdated
|
|
||
| ``pyflyte run`` is useful when you are getting started with Flyte, testing small scripts, or iterating over local scripts. | ||
|
|
||
| ``pyflyte package + flytectl register`` is useful when you wish to work with multiple admins, wherein you can package the script, compile it into protobuf objects, write it to local disk, and upload this zipped package to different FlyteAdmins. |
Signed-off-by: SmritiSatyanV [email protected]
wild-endeavor
approved these changes
Aug 13, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue
Document
pyflyte registerandpyflyte runSigned-off-by: SmritiSatyanV [email protected]