Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

updated pyflyte documentation #2877

Open
wants to merge 9 commits into
base: master
Choose a base branch
from
Open

updated pyflyte documentation #2877

Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
8b43094
updated error message for --pkgs option
sumana-2705 Oct 30, 2024
0d8127b
added example usage of --source option in help message
sumana-2705 Oct 30, 2024
5999651
added a structure explaining clear usage of --pkgs option in main hel…
sumana-2705 Oct 30, 2024
c28bec2
fixed linting errors
sumana-2705 Oct 30, 2024
58e4008
updated package.py file according to the review
sumana-2705 Nov 7, 2024
eb8ebd4
undo the changes in register.py file
sumana-2705 Nov 7, 2024
f9400cf
Merge branch 'flyteorg:master' into pyflyte-docs-fix
sumana-2705 Nov 15, 2024
090a118
updated comments
sumana-2705 Nov 15, 2024
3baa154
updated clone
sumana-2705 Nov 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion flytekit/clis/sdk_in_container/package.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@
required=False,
type=click.Path(exists=True, file_okay=False, readable=True, resolve_path=True, allow_dash=True),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have a minimal working example that shows how the --source argument of pyflyte package is supposed to be used? I'm asking because I ever only use the --pkgs arg and I don't know in which situation it makes sense to use the --source arg. I feel as part of this issue the difference should become clear from the help messages.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fg91 I have no idea about working example, will this help message works?

help="Local filesystem path to the root of the package. Use --source when specifying the root directory of your workflows. Example: --source /path/to/workflows. Use --pkgs to specify package names directly.",

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's useful but will let @fg91 comment on this one

Copy link
Member

@fg91 fg91 Nov 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I unfortunately also don't have an idea how the --source tag is supposed to be used over the --pkgs arg.

My proposition would be to merge this PR as is (after this tiny nit has been merged) but to leave the issue open and then ask in Flyte's ask-the-community channel whether anyone knows. If yes, we can make a second PR explaining the diff. If there isn't any real use for the --source arg, we can create an issue to deprecate it.
What do you think about this?

default=".",
help="Local filesystem path to the root of the package.",
help="Local filesystem path to the root of the package. Example: --source /path/to/workflows",
)
@click.option(
"-o",
Expand Down Expand Up @@ -129,6 +129,20 @@ def package(
For workflows, one pb file is produced for each workflow, representing a WorkflowClosure object. The closure
object contains the WorkflowTemplate, along with the relevant tasks for that workflow.
This serialization step will set the name of the tasks to the fully qualified name of the task function.

Given a Python package containing Flyte entities with a structure like this ...

.
└── some_package
├── __init__.py
└── some_module
├── __init__.py
└── wf.py

... you can package the Flyte entities contained in this Python package by executing the following command:
pyflyte --pkgs some_package package

This command parses and compiles the user's Python code into Flyte protobuf objects that can be registered with the backend.
"""
# Ensure that the two flags are consistent
if fast:
Expand Down