feat: Support ImageSpec as base image

[pingsutw]

Tracking issue

NA

Why are the changes needed?

It enables users to easily reuse the ImageSpec and merge different ImageSpecs together.

What changes were proposed in this pull request?

ImageBuildEngine will build the base image if the base image is an ImageSpec.

How was this patch tested?

pyflyte run --remote ..

Setup process

from flytekit import task, workflow, ImageSpec, dynamic

new_flytekit = "git+https://github.com/flyteorg/flytekit@64730c8ffb4e76cb268817e30d31c6d209452e75"
image_sklearn = ImageSpec(packages=["scikit-learn", new_flytekit], apt_packages=["git"], registry="pingsutw")
image_tensorflow = ImageSpec(base_image=image_sklearn, packages=["tensorflow"], registry="pingsutw")


@task(container_image=image_sklearn)
def t1(a: int) -> int:
    return a + 2


@workflow
def wf(a: int = 3):
    t1(a=a)


if __name__ == "__main__":
    print(f"Running my_wf: {wf(a=50)}")

Screenshots

Check all the applicable boxes

• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.

Related PRs

NA

Docs link

NA

[codecov]

Codecov Report

Attention: Patch coverage is 44.44444% with 5 lines in your changes are missing coverage. Please review.

Project coverage is 83.31%. Comparing base (3f45131) to head (edbaf85).
Report is 4 commits behind head on master.

Files	Patch %	Lines
flytekit/image_spec/image_spec.py	16.66%	3 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2277      +/-   ##
==========================================
- Coverage   83.79%   83.31%   -0.49%     
==========================================
  Files         332      309      -23     
  Lines       25147    24057    -1090     
  Branches     3703     3493     -210     
==========================================
- Hits        21071    20042    -1029     
+ Misses       3450     3387      -63     
- Partials      626      628       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pingsutw]

cc @cjidboon94 This PR can address your issue as well. you can have an ImageSpec like

image1 = ImageSpec(packages=["scikit-learn"], pip_index=<index1>)
image2 = ImageSpec(base_image=image1, packages=["tensorflow"], pip_index=<index2>)

[cjidboon94]

cc @cjidboon94 This PR can address your issue as well. you can have an ImageSpec like

image1 = ImageSpec(packages=["scikit-learn"], pip_index=<index1>)
image2 = ImageSpec(base_image=image1, packages=["tensorflow"], pip_index=<index2>)

While it would solve it, it feels wasteful if image1 is not going to be reused by other tasks/imagespecs definitions (this would be the case for me).

[Future-Outlier]

TEST