Skip to content
This repository has been archived by the owner on Oct 9, 2023. It is now read-only.

Remove resource injection on the node for container task #544

Merged
merged 4 commits into from
Apr 11, 2023

Conversation

ByronHsu
Copy link
Contributor

@ByronHsu ByronHsu commented Mar 27, 2023

TL;DR

Remove resource injection on the node for container task

Type

  • Bug Fix
  • Feature
  • Plugin

Are all requirements met?

  • Code completed
  • Smoke tested
  • Unit tests added
  • Code documentation added
  • Any pending items have an associated Issue

Complete description

How did you fix the bug, make the feature etc. Link to any design docs etc

In BuildRawContainer, we do not inject the container resource from the task template. Instead, we inject the resource from parameters.TaskExecMetadata.GetOverrides().GetResources().

This is because for both _with_override(resource=... and @task(requests=...), they are both inject as the node resource override in buildNodeSpec

Expected behavior

For the case of @task(requests=...), the resource override on the node should be empty, and the resource is injected from tasktemplate at BuildRawContainer

Tracking Issue

flyteorg/flyte#3540

@codecov
Copy link

codecov bot commented Mar 27, 2023

Codecov Report

Merging #544 (e6c4e03) into master (7abdcd7) will increase coverage by 0.40%.
The diff coverage is n/a.

❗ Current head e6c4e03 differs from pull request most recent head 72e2f1d. Consider uploading reports for the commit 72e2f1d to get more accurate results

@EngHabu
Copy link
Contributor

EngHabu commented Mar 27, 2023

I'm probably missing something but should we be trying to do that here? https://github.com/flyteorg/flyteplugins/blob/master/go/tasks/pluginmachinery/flytek8s/container_helper.go#L305

What issue are you seeing, @ByronHsu ?

@ByronHsu
Copy link
Contributor Author

@EngHabu Thanks for the quick reply. Yes, I have another PR injecting the resource there. See the attachment in the PR flyteorg/flyte#3540.
No issue for the OSS version, but we are hacking something internally, and this logic flaw is blocking us.

@ByronHsu ByronHsu force-pushed the gh/container-resource branch from e57edbf to e377136 Compare March 29, 2023 05:54
@ByronHsu ByronHsu force-pushed the gh/container-resource branch from 2fa2518 to 9efb52f Compare April 9, 2023 05:48
Signed-off-by: byhsu <[email protected]>
go.mod Outdated Show resolved Hide resolved
Signed-off-by: byhsu <[email protected]>
@hamersaw hamersaw merged commit ef164c5 into flyteorg:master Apr 11, 2023
eapolinario pushed a commit to eapolinario/flytepropeller that referenced this pull request Aug 9, 2023
* Remove resource injection on the node for container task

Signed-off-by: byhsu <[email protected]>

* fix conflict

Signed-off-by: byhsu <[email protected]>

* fix lint

Signed-off-by: byhsu <[email protected]>

* downgrade plugin

Signed-off-by: byhsu <[email protected]>

---------

Signed-off-by: byhsu <[email protected]>
Co-authored-by: byhsu <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants