Code

[release v1.2] Backport v1.6#1645

Code

Merged

eapolinario merged 81 commits intorelease-v1.2flyteorg/flytekit:release-v1.2from backport-1.6-to-release-v1.2flyteorg/flytekit:backport-1.6-to-release-v1.2Copy head branch name to clipboardMay 17, 2023

+6,599-2,791

Commits

Commits on May 16, 2023

• Data subsystem (#1526)

  Show description for 4555578
  

  wild-endeavor

  authored and

  eapolinario

  committed
  4555578

• Pluck retry from flytekit and into sagemaker (#1411)

  Show description for f6331ca
  eapolinario

  committed
  f6331ca

• Update the pypi wait (#1554)

  Show description for 2731766
  

  wild-endeavor

  authored and

  eapolinario

  committed
  2731766

• Stream Directories and Files using Flyte (#1512)

  Show description for 0c47d46
  

  kumare3

  authored and

  eapolinario

  committed
  0c47d46

• Make FlyteFile compatible with Annotated[..., HashMethod] (#1544)

  Show description for 1f7f510
  

  AdrianoKF

  authored and

  eapolinario

  committed
  1f7f510

• move FlyteSchema deprecation warning to initialization method (#1558)

  Show description for 236309a
  

  cosmicBboy

  authored and

  eapolinario

  committed
  236309a

• add pod_template and pod_template_name arguments for ContainerTask (#1515)

  Show description for 756148b
  

  2 peopleauthored and

  eapolinario

  committed
  756148b

• Pass locally defined scopes to RemoteClientConfigStore (#1553)

  Show description for b22e5e3
  

  franco-bocci

  authored and

  eapolinario

  committed
  b22e5e3

• TypeTransformer for TensorFlow model (#1562)

  Show description for f0abcc9
  

  samhita-alla

  authored and

  eapolinario

  committed
  f0abcc9

• Remove fsspec flytekit plugin from main Dockerfile (#1561)

  Show description for bd6a7e5
  

  wild-endeavor

  authored and

  eapolinario

  committed
  bd6a7e5

• Device auth flow / Headless auth (#1552)

  Show description for b757306
  

  kumare3

  authored and

  eapolinario

  committed
  b757306

• url encode secret in client credentials flow (#1566)

  Show description for 7e76d4b
  

  wild-endeavor

  authored and

  eapolinario

  committed
  7e76d4b

• Python run multiple files (#1559)

  Show description for 19a0367
  

  pingsutw

  authored and

  eapolinario

  committed
  19a0367

• General Partial support in flytekit and multi-list support in flytekit (#1556)

  Show description for 9a31a01
  

  kumare3

  authored and

  eapolinario

  committed
  9a31a01

• fix: Silence keyring warnings by changing to debug (#1568)

  Show description for d8c10c6
  

  2 peopleauthored and

  eapolinario

  committed
  d8c10c6

• Support GCP secrets (#1571)

  Show description for 06ea83b
  

  wild-endeavor

  authored and

  eapolinario

  committed
  06ea83b

• Automatically remove unused import (#1574)

  Show description for 57e21c2
  

  pingsutw

  authored and

  eapolinario

  committed
  57e21c2

• Disallow partial lists in map tasks (#1577)

  Show description for 082f8fb
  eapolinario

  committed
  082f8fb

• Remove duplicate reporting logic (#1578)

  Show description for 0c0c2ac
  

  wild-endeavor

  authored and

  eapolinario

  committed
  0c0c2ac

• [Core feature] Convert List[Any] to a single pickle file (#1535)

  Show description for 3ed7f83
  

  3 peopleauthored and

  eapolinario

  committed
  3ed7f83

• Improve authoring structure documentation (#1572)

  Show description for 8941718
  

  samhita-alla

  authored and

  eapolinario

  committed
  8941718

• Unify sqlalchemy Dockerfiles (#1585)

  Show description for 3be7f50
  eapolinario

  committed
  3be7f50

• Pyflyte build imageSpec (#1555)

  Show description for a591b7d
  

  pingsutw

  authored and

  eapolinario

  committed
  a591b7d

• Feature/3506: Improve Type Conversion errors, use rich to prettify error messages (#1582)

  Show description for 915ea43
  

  cosmicBboy

  authored and

  eapolinario

  committed
  915ea43

• Add support nested FlyteFile in pyflyte run (#1587)

  Show description for f8cc630
  

  pingsutw

  authored and

  eapolinario

  committed
  f8cc630

• Set cache_regions=True in the case of s3fs (#1592)

  Show description for 8e26a59
  eapolinario

  committed
  8e26a59

• Set default ComSpec when running on Windows (#1595)

  Show description for b60fe84
  

  pingsutw

  authored and

  eapolinario

  committed
  b60fe84

• Feature: add activate-launchplan command to pyflyte (#1588)

  Show description for ec61f34
  

  wckdman

  authored and

  eapolinario

  committed
  ec61f34

• nit (#1596)

  Show description for 34ea031
  

  wild-endeavor

  authored and

  eapolinario

  committed
  34ea031

• Add log streaming to papermill plugin (#1129)

  Show description for 34326ca
  

  3 peopleauthored and

  eapolinario

  committed
  34326ca

• Set lower bound for s3fs (#1598)

  Show description for a545e9e
  

  pingsutw

  authored and

  eapolinario

  committed
  a545e9e

• Implement horovod task in mpi plugin (#1575)

  Show description for 083b16b
  

  ByronHsu

  authored and

  eapolinario

  committed
  083b16b

• add ImageRender class for Flyte Decks (#1599)

  Show description for e3447ce
  

  peridotml

  authored and

  eapolinario

  committed
  e3447ce

• Bump tensorflow from 2.10.0 to 2.11.1 in /plugins/flytekit-mlflow (#1586)

  Show description for 2d6aaa5
  

  dependabot[bot]

  authored and

  eapolinario

  committed
  2d6aaa5

• Handle batchable lists of length 0 (#1600)

  Show description for 5d4da60
  eapolinario

  committed
  5d4da60

• pyflyte run imperative workflow (#1597)

  Show description for 5097ee7
  

  pingsutw

  authored and

  eapolinario

  committed
  5097ee7

• Backfill should fill in the right input vars (#1593)

  Show description for ac92ce6
  

  kumare3

  authored and

  eapolinario

  committed
  ac92ce6

• Pyflyte prettified (#1602)

  Show description for 66a3ee6
  

  kumare3

  authored and

  eapolinario

  committed
  66a3ee6

• pyflyte run now supports json/yaml files (#1606)

  Show description for 21bd0a7
  

  kumare3

  authored and

  eapolinario

  committed
  21bd0a7

• Add is_inside func to ImageSpec (#1601)

  Show description for 93179b1
  

  pingsutw

  authored and

  eapolinario

  committed
  93179b1

• Measuring the time taken for each component when executing a task (#1581)

  Show description for 3eaca9c
  

  Yicheng-Lu-llll

  authored and

  eapolinario

  committed
  3eaca9c

• Skip the image building process if the check for its existence fails (#1614)

  Show description for 1df828f
  

  pingsutw

  authored and

  eapolinario

  committed
  1df828f

• Enable torch elastic training (torchrun) (#1603)

  Show description for c7f20e6
  

  fg91

  authored and

  eapolinario

  committed
  c7f20e6

• update sphinx youtube plugin (#1619)

  Show description for 33a2f32
  

  pingsutw

  authored and

  eapolinario

  committed
  33a2f32

• Read polar dataframe without copying to local (#1618)

  Show description for b2fb4bf
  

  pingsutw

  authored and

  eapolinario

  committed
  b2fb4bf

• Add default spark image (#1616)

  Show description for d778232
  

  pingsutw

  authored and

  eapolinario

  committed
  d778232

• Lazy load modules (#1590)

  Show description for 5a66411
  

  pingsutw

  authored and

  eapolinario

  committed
  5a66411

• Override node name with a name containing underscore (#1608)

  Show description for 7c64c8d
  

  pingsutw

  authored and

  eapolinario

  committed
  7c64c8d

• External Plugin Service (grpc) (#1524)

  Show description for a04a128
  

  pingsutw

  authored and

  eapolinario

  committed
  a04a128

• Include traceback in errors from admin (#1621)

  Show description for d9174fd
  

  wild-endeavor

  authored and

  eapolinario

  committed
  d9174fd

• Add support for copying all the files in source root (#1622)

  Show description for 6577c61
  

  pingsutw

  authored and

  eapolinario

  committed
  6577c61

• fix PipelineModel transformer issue 3648 (#1623)

  Show description for 4c23c91
  

  peridotml

  authored and

  eapolinario

  committed
  4c23c91

• update GH workflow for external plugin service (#1624)

  Show description for a7391c9
  

  pingsutw

  authored and

  eapolinario

  committed
  a7391c9

• [Issue-3617] Enables FlyteFiles, FlyteDirectors, and StructuredDatasets inputs in papermill plugin (#1612)

  

  peridotml

  authored and

  eapolinario

  committed
  7409889

• Add support for using a list as an input for a subworkflow (#1605)

  Show description for 396080c
  

  pingsutw

  authored and

  eapolinario

  committed
  396080c

• Improve task decorator type hints with overload (#1631)

  Show description for 2fab5db
  

  ringohoffman

  authored and

  eapolinario

  committed
  2fab5db

• add annotation option for serialization (#1615)

  Show description for af17385
  

  3 peopleauthored and

  eapolinario

  committed
  af17385

• Delete removed data persistence classes from docs (#1633)

  Show description for fc065db
  

  cosmicBboy

  authored and

  eapolinario

  committed
  fc065db

• Improve workflow decorator type hints with overload (#1635)

  Show description for ded15a8
  

  ringohoffman

  authored and

  eapolinario

  committed
  ded15a8

• Fix mypy errors (#1313)

  Show description for af49155
  

  2 peopleauthored and

  eapolinario

  committed
  af49155

• type-protobuf<4

  Show description for 725cdb7
  eapolinario

  committed
  725cdb7

Commits on May 17, 2023

• Allow for python 3.7

  Show description for 02f7830
  eapolinario

  committed
  02f7830

• Fix protobuf transformer merge error

  eapolinario

  committed
  6836d62

• Fix bad merge of s/_task_models/task_models/

  eapolinario

  committed
  bbda9ad

• Add maxsize=128 to @lru_cache to cover python 3.7

  Show description for 8914849
  eapolinario

  committed
  8914849

• Lint

  Show description for 2178128
  eapolinario

  committed
  2178128

• Regenerate doc-requirements.txt

  Show description for 6059a60
  eapolinario

  committed
  6059a60

• Add missing call to lru_cache

  Show description for f5e4ae5
  eapolinario

  committed
  f5e4ae5

• Bump version and regenerate requirements in a few plugins

  Show description for f24f261
  eapolinario

  committed
  f24f261

• Revert "Fix mypy errors (#1313)"

  Show description for 14fb5b5
  eapolinario

  committed
  14fb5b5

• Remove docs

  Show description for 184b173
  eapolinario

  committed
  184b173

• More linting

  Show description for 61d0afc
  eapolinario

  committed
  61d0afc

• Fix type of get_type_for_output_var

  Show description for 5ee2067
  eapolinario

  committed
  5ee2067

• Remove tensorflow test_model

  eapolinario

  committed
  2fd8094

• Replace use of shutil.copytree

  Show description for a947a45
  eapolinario

  committed
  a947a45

• Other use of copytree

  Show description for e36e4f6
  eapolinario

  committed
  e36e4f6

• Stringify paths in test_script_mode

  Show description for d5547e7
  eapolinario

  committed
  d5547e7

• os.path.isfile instead in imagespec

  Show description for 7a01c0a
  eapolinario

  committed
  7a01c0a

• Remove tests

  Show description for f1c71bb
  eapolinario

  committed
  f1c71bb

• More linting

  Show description for d10cac7
  eapolinario

  committed
  d10cac7

• Remove flaky test

  Show description for 1540d1c
  eapolinario

  committed
  1540d1c