Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split service code #2136

Merged
merged 74 commits into from
Jan 31, 2024
Merged

Split service code #2136

merged 74 commits into from
Jan 31, 2024

Conversation

wild-endeavor
Copy link
Contributor

@wild-endeavor wild-endeavor commented Jan 30, 2024

Split out the implementation of the service for now.

thomasjpfan and others added 30 commits December 22, 2023 13:17
* Error when multiple configuration plugins are installed

Signed-off-by: Thomas J. Fan <[email protected]>

* Adds semi-colon

Signed-off-by: Thomas J. Fan <[email protected]>

---------

Signed-off-by: Thomas J. Fan <[email protected]>
Signed-off-by: Future Outlier <[email protected]>
Co-authored-by: Future Outlier <[email protected]>
* added _cmd_prefix handling

Signed-off-by: Daniel Rammer <[email protected]>

* fixed typing imports

Signed-off-by: Daniel Rammer <[email protected]>

* add get_config

Signed-off-by: Kevin Su <[email protected]>

* updating get_config to use underlying functions config

Signed-off-by: Daniel Rammer <[email protected]>

---------

Signed-off-by: Daniel Rammer <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Co-authored-by: Kevin Su <[email protected]>
* Accelerator docs

Signed-off-by: Ketan Umare <[email protected]>

* updating docs

Signed-off-by: Ketan Umare <[email protected]>

* Updated docs

Signed-off-by: Ketan Umare <[email protected]>

* updated docs

Signed-off-by: Ketan Umare <[email protected]>

* more docs update

Signed-off-by: Ketan Umare <[email protected]>

* updated

Signed-off-by: Ketan Umare <[email protected]>

* updated docs

Signed-off-by: Ketan Umare <[email protected]>

---------

Signed-off-by: Ketan Umare <[email protected]>
Co-authored-by: Ketan Umare <[email protected]>
* Add support for running remote-tasks and remote-workflows

Signed-off-by: Ketan Umare <[email protected]>

* updated

Signed-off-by: Ketan Umare <[email protected]>

---------

Signed-off-by: Ketan Umare <[email protected]>
Co-authored-by: Ketan Umare <[email protected]>
Bumps [fonttools](https://github.com/fonttools/fonttools) from 4.41.1 to 4.43.0.
- [Release notes](https://github.com/fonttools/fonttools/releases)
- [Changelog](https://github.com/fonttools/fonttools/blob/main/NEWS.rst)
- [Commits](fonttools/fonttools@4.41.1...4.43.0)

---
updated-dependencies:
- dependency-name: fonttools
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* add monodocs index page in flytekit

Signed-off-by: Niels Bantilan <[email protected]>

* updates

Signed-off-by: Niels Bantilan <[email protected]>

---------

Signed-off-by: Niels Bantilan <[email protected]>
* add monodocs redirects to the flytekit readthedocs project

Signed-off-by: Niels Bantilan <[email protected]>

* update requirements

Signed-off-by: Niels Bantilan <[email protected]>

* add redirects enabling flag

Signed-off-by: Niels Bantilan <[email protected]>

* update

Signed-off-by: Niels Bantilan <[email protected]>

---------

Signed-off-by: Niels Bantilan <[email protected]>
Signed-off-by: Future Outlier <[email protected]>
Co-authored-by: Future Outlier <[email protected]>
Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.2 to 3.1.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.2...3.1.3)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.2 to 3.1.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.2...3.1.3)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Future Outlier <[email protected]>
Co-authored-by: Future Outlier <[email protected]>
)

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.2 to 3.1.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.2...3.1.3)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.2 to 3.1.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.2...3.1.3)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Future Outlier <[email protected]>
Co-authored-by: Future Outlier <[email protected]>
Signed-off-by: Future Outlier <[email protected]>
Co-authored-by: Future Outlier <[email protected]>
@wild-endeavor wild-endeavor changed the title Artf/split engine Split service code Jan 30, 2024
wild-endeavor and others added 3 commits January 29, 2024 16:03
Signed-off-by: Yee Hing Tong <[email protected]>
…2127)

Bumps [pillow](https://github.com/python-pillow/Pillow) from 10.1.0 to 10.2.0.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](python-pillow/Pillow@10.1.0...10.2.0)

---
updated-dependencies:
- dependency-name: pillow
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Yee Hing Tong <[email protected]>
Copy link

codecov bot commented Jan 30, 2024

Codecov Report

Attention: 72 lines in your changes are missing coverage. Please review.

Comparison is base (63b7b29) 53.53% compared to head (cb14099) 75.11%.

Files Patch % Lines
flytekit/core/artifact.py 34.28% 46 Missing ⚠️
flytekit/tools/repo.py 42.85% 7 Missing and 1 partial ⚠️
flytekit/extend/backend/base_agent.py 64.28% 3 Missing and 2 partials ⚠️
flytekit/core/type_engine.py 63.63% 2 Missing and 2 partials ⚠️
flytekit/image_spec/image_spec.py 85.71% 3 Missing and 1 partial ⚠️
flytekit/extend/backend/agent_service.py 57.14% 3 Missing ⚠️
flytekit/core/interface.py 0.00% 1 Missing ⚠️
flytekit/remote/remote.py 50.00% 1 Missing ⚠️
Additional details and impacted files
@@              Coverage Diff               @@
##           artifacts    #2136       +/-   ##
==============================================
+ Coverage      53.53%   75.11%   +21.57%     
==============================================
  Files            178      177        -1     
  Lines          17617    17482      -135     
  Branches        3656     3618       -38     
==============================================
+ Hits            9431    13131     +3700     
+ Misses          7755     3742     -4013     
- Partials         431      609      +178     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

dependabot bot and others added 18 commits January 30, 2024 00:41
Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.8.6 to 3.9.2.
- [Release notes](https://github.com/aio-libs/aiohttp/releases)
- [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst)
- [Commits](aio-libs/aiohttp@v3.8.6...v3.9.2)

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.9.1 to 3.9.2.
- [Release notes](https://github.com/aio-libs/aiohttp/releases)
- [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst)
- [Commits](aio-libs/aiohttp@v3.9.1...v3.9.2)

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.9.0 to 3.9.2.
- [Release notes](https://github.com/aio-libs/aiohttp/releases)
- [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst)
- [Commits](aio-libs/aiohttp@v3.9.0...v3.9.2)

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.9.1 to 3.9.2.
- [Release notes](https://github.com/aio-libs/aiohttp/releases)
- [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst)
- [Commits](aio-libs/aiohttp@v3.9.1...v3.9.2)

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
#2126)

Bumps [pillow](https://github.com/python-pillow/Pillow) from 10.1.0 to 10.2.0.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](python-pillow/Pillow@10.1.0...10.2.0)

---
updated-dependencies:
- dependency-name: pillow
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
* Pass environment variables to local executions

Signed-off-by: Eduardo Apolinario <[email protected]>

* Pass environment variables to local executions

Signed-off-by: Eduardo Apolinario <[email protected]>

* Fix test_get_entities_in_file

Signed-off-by: Eduardo Apolinario <[email protected]>

* Lint

Signed-off-by: Eduardo Apolinario <[email protected]>

---------

Signed-off-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>
* Add output_entity_hint and support for serialising it

Signed-off-by: Thomas Newton <[email protected]>

* Correct assertion

Signed-off-by: Thomas Newton <[email protected]>

* Add output_entity_hint to task decorator

Signed-off-by: Thomas Newton <[email protected]>

* Type hints

Signed-off-by: Thomas Newton <[email protected]>

* Fix circular import of type hints

Signed-off-by: Thomas Newton <[email protected]>

* Support iterable of output_entity_hints (#5)

Signed-off-by: Thomas Newton <[email protected]>

* Fix circular import for type hints

Signed-off-by: Thomas Newton <[email protected]>

* Fix tests

Signed-off-by: Thomas Newton <[email protected]>

* Auto-format

Signed-off-by: Thomas Newton <[email protected]>

* Add positive test case

Signed-off-by: Thomas Newton <[email protected]>

* Add a test for disallowing entity hints on static tasks

Signed-off-by: Thomas Newton <[email protected]>

* raise instead of assert

Signed-off-by: Thomas Newton <[email protected]>

* Rename output_entity_hints -> node_dependency_hints

Signed-off-by: Thomas Newton <[email protected]>

* Update docstrings

Signed-off-by: Thomas Newton <[email protected]>

* Auto-format

Signed-off-by: Thomas Newton <[email protected]>

* Fix test_serialize_vscode

Signed-off-by: Thomas Newton <[email protected]>

* Maybe fix the docs error

Signed-off-by: Thomas Newton <[email protected]>

* Fix a newly added test from master

Signed-off-by: Thomas Newton <[email protected]>

* Fix monodocs build

Signed-off-by: Thomas Newton <[email protected]>

---------

Signed-off-by: Thomas Newton <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>
* Warn user when overriding requests but not limits

Signed-off-by: Fabio Grätz <[email protected]>

* Lint

Signed-off-by: Fabio Grätz <[email protected]>

---------

Signed-off-by: Fabio Grätz <[email protected]>
Co-authored-by: Fabio Grätz <[email protected]>
Signed-off-by: Future Outlier <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Co-authored-by: Future Outlier <[email protected]>
Co-authored-by: Kevin Su <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
…ng fails (#2047)

* Improve error message in workflow compliation when output binding fails

Signed-off-by: Fabio Graetz <[email protected]>

* Include previous exception in error message

Signed-off-by: Fabio Graetz <[email protected]>

* Adapt tests

Signed-off-by: Fabio Graetz <[email protected]>

* Add test that would have caught issue

Signed-off-by: Fabio Graetz <[email protected]>

* Silence expected mypy warning in test

Signed-off-by: Fabio Graetz <[email protected]>

* Use FlyteValidationException instead of AssertionError

Signed-off-by: Fabio Grätz <[email protected]>

* Use FlyteValidationException instead of AssertionError

Signed-off-by: Fabio Grätz <[email protected]>

* Lint

Signed-off-by: Fabio Grätz <[email protected]>

---------

Signed-off-by: Fabio Graetz <[email protected]>
Signed-off-by: Fabio Grätz <[email protected]>
Co-authored-by: Fabio Grätz <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
@wild-endeavor wild-endeavor merged commit e76efdc into artifacts Jan 31, 2024
23 of 29 checks passed
eapolinario added a commit that referenced this pull request Feb 8, 2024
* temp

Signed-off-by: Yee Hing Tong <[email protected]>

* stuff

Signed-off-by: Yee Hing Tong <[email protected]>

* temp

Signed-off-by: Yee Hing Tong <[email protected]>

* scaffolding areas mostly identified

Signed-off-by: Yee Hing Tong <[email protected]>

* add artifact to upload request

Signed-off-by: Yee Hing Tong <[email protected]>

* remove an unnecessary line in workflow

Signed-off-by: Yee Hing Tong <[email protected]>

* finish adding examples use cases maybe

Signed-off-by: Yee Hing Tong <[email protected]>

* add project/dom to get query

Signed-off-by: Yee Hing Tong <[email protected]>

* add from flyte idl

Signed-off-by: Yee Hing Tong <[email protected]>

* add project domain to as query

Signed-off-by: Yee Hing Tong <[email protected]>

* add condition in parameter to flyte idl

Signed-off-by: Yee Hing Tong <[email protected]>

* test stuff

* Remove artifactID from literal oneof, add to metadata (#2)

* Triggers (#6)

* Minor changes to get time series example working #8

Signed-off-by: Yee Hing Tong <[email protected]>

* switch channel (#10)

Signed-off-by: Yee Hing Tong <[email protected]>

* fix tests ignore - pr into other pr (#1858)

Signed-off-by: Yee Hing Tong <[email protected]>

* Artf/update idl ux (#1920)


Signed-off-by: Yee Hing Tong <[email protected]>

* Artf/trigger (#1948)

* Add triggers
* Remove bind_partition_time and just assume users won't use that. It's just time_partition in the normal call function now.

Signed-off-by: Yee Hing Tong <[email protected]>

* remove the now deleted artifact spec (#1984)

Signed-off-by: Yee Hing Tong <[email protected]>

* Literal metadata model update (#2089)


Signed-off-by: Yee Hing Tong <[email protected]>

* Separate time partition (#2114)



Signed-off-by: Yee Hing Tong <[email protected]>

* Split service code (#2136)


Signed-off-by: Yee Hing Tong <[email protected]>

* remove empty files

Signed-off-by: Yee Hing Tong <[email protected]>

* add noneness check to metadata and add test

Signed-off-by: Yee Hing Tong <[email protected]>

* remove sandbox test for now

Signed-off-by: Yee Hing Tong <[email protected]>

* Artf/cleanup (#2158)

* add a test

Signed-off-by: Yee Hing Tong <[email protected]>

* try updates

Signed-off-by: Yee Hing Tong <[email protected]>

---------

Signed-off-by: Yee Hing Tong <[email protected]>

* Use python 3.9 to run make doc-requirements.txt

Signed-off-by: Eduardo Apolinario <[email protected]>

* reasons not msg

Signed-off-by: Yee Hing Tong <[email protected]>

---------

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>
eapolinario added a commit that referenced this pull request Feb 8, 2024
* temp

Signed-off-by: Yee Hing Tong <[email protected]>

* stuff

Signed-off-by: Yee Hing Tong <[email protected]>

* temp

Signed-off-by: Yee Hing Tong <[email protected]>

* scaffolding areas mostly identified

Signed-off-by: Yee Hing Tong <[email protected]>

* add artifact to upload request

Signed-off-by: Yee Hing Tong <[email protected]>

* remove an unnecessary line in workflow

Signed-off-by: Yee Hing Tong <[email protected]>

* finish adding examples use cases maybe

Signed-off-by: Yee Hing Tong <[email protected]>

* add project/dom to get query

Signed-off-by: Yee Hing Tong <[email protected]>

* add from flyte idl

Signed-off-by: Yee Hing Tong <[email protected]>

* add project domain to as query

Signed-off-by: Yee Hing Tong <[email protected]>

* add condition in parameter to flyte idl

Signed-off-by: Yee Hing Tong <[email protected]>

* test stuff

* Remove artifactID from literal oneof, add to metadata (#2)

* Triggers (#6)

* Minor changes to get time series example working #8

Signed-off-by: Yee Hing Tong <[email protected]>

* switch channel (#10)

Signed-off-by: Yee Hing Tong <[email protected]>

* fix tests ignore - pr into other pr (#1858)

Signed-off-by: Yee Hing Tong <[email protected]>

* Artf/update idl ux (#1920)

Signed-off-by: Yee Hing Tong <[email protected]>

* Artf/trigger (#1948)

* Add triggers
* Remove bind_partition_time and just assume users won't use that. It's just time_partition in the normal call function now.

Signed-off-by: Yee Hing Tong <[email protected]>

* remove the now deleted artifact spec (#1984)

Signed-off-by: Yee Hing Tong <[email protected]>

* Literal metadata model update (#2089)

Signed-off-by: Yee Hing Tong <[email protected]>

* Separate time partition (#2114)

Signed-off-by: Yee Hing Tong <[email protected]>

* Split service code (#2136)

Signed-off-by: Yee Hing Tong <[email protected]>

* remove empty files

Signed-off-by: Yee Hing Tong <[email protected]>

* add noneness check to metadata and add test

Signed-off-by: Yee Hing Tong <[email protected]>

* remove sandbox test for now

Signed-off-by: Yee Hing Tong <[email protected]>

* Artf/cleanup (#2158)

* add a test

Signed-off-by: Yee Hing Tong <[email protected]>

* try updates

Signed-off-by: Yee Hing Tong <[email protected]>

---------

Signed-off-by: Yee Hing Tong <[email protected]>

* Use python 3.9 to run make doc-requirements.txt

Signed-off-by: Eduardo Apolinario <[email protected]>

* reasons not msg

Signed-off-by: Yee Hing Tong <[email protected]>

---------

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.