Fix Flyin Plugin VSCode download bug

[Future-Outlier]

Tracking issue

Fixes flyteorg/flyte#4284

Why are the changes needed?

We don't support cases when using default flyin image and we want to install additional extensions.
https://github.com/flyteorg/flytekit/blob/master/plugins/flytekit-flyin/Dockerfile
For example, you use COPILOT_CONFIG, which include python, jupyter and copilot extensions in the list.
The old code will not install the copilot extensions for 2 reasons:

1. permission denied for flytekit user in the container
2. there's a bug in the decorator.py, which will skip downloading additional extensions if code-server is executable in the beginning

How was this patch tested?

unit test and in single binary mode.

Setup process

0. start the flyte single binary mode
1. build an image by Dockerfile
2. print the log in the container and see if the copilot extension is installed or not

docker buildx build --platform=linux/amd64 -t localhost:30000/flyin-0112:1536 -f DockerfileFlyin_v2 .
docker push localhost:30000/flyin-0112:1536
pyflyte run --remote --image localhost:30000/flyin-0112:1536 vscode_task.py wf

FROM python:3.10-slim-buster

MAINTAINER Flyte Team <[email protected]>
LABEL org.opencontainers.image.source https://github.com/flyteorg/flytekit
WORKDIR /root
ENV PYTHONPATH /root

ARG VERSION
ARG TARGETARCH

# 1. Update the necessary packages for flytekit
# 2. Install code-server
# 3. Download code-server extensions for Python and Jupyter via wget
# 4. Install flytekit and flytekit-flyin with no cache
# 5. Delete apt cache. Reference: https://gist.github.com/marvell/7c812736565928e602c4
# 6. Some packages will create config file under /home by default, so we need to make sure it's writable
# 7. Change the permission of /tmp, so that others can run command on it
RUN apt-get update \
    && apt-get install build-essential git wget -y \
    && mkdir -p /tmp/ \
    && mkdir -p /tmp/code-server \
    && wget --no-check-certificate -O /tmp/code-server/code-server-4.19.0-linux-${TARGETARCH}.tar.gz https://github.com/coder/code-server/releases/download/v4.19.0/code-server-4.19.0-linux-${TARGETARCH}.tar.gz \
    && tar -xzf /tmp/code-server/code-server-4.19.0-linux-${TARGETARCH}.tar.gz -C /tmp/code-server/ \
    && wget --no-check-certificate https://open-vsx.org/api/ms-python/python/2023.20.0/file/ms-python.python-2023.20.0.vsix -P /tmp/code-server \
    && wget --no-check-certificate https://open-vsx.org/api/ms-toolsai/jupyter/2023.9.100/file/ms-toolsai.jupyter-2023.9.100.vsix -P /tmp/code-server 
RUN pip install -U git+https://github.com/Future-Outlier/flytekit.git@da29c20f7d7d43e6a7491a689cd685d18be9de8d \
    && pip install -U git+https://github.com/Future-Outlier/flytekit.git@da29c20f7d7d43e6a7491a689cd685d18be9de8d#subdirectory=plugins/flytekit-flyin \
    && apt-get clean autoclean \
    && apt-get autoremove --yes \
    && rm -rf /var/lib/{apt,dpkg,cache,log}/ \
    && useradd -u 1000 flytekit \
    && chown -R flytekit:flytekit /tmp/code-server \
    && chown flytekit: /root \
    && chown flytekit: /home \
    && :

# Set the environment variable for code-server
ENV PATH="/tmp/code-server/code-server-4.19.0-linux-${TARGETARCH}/bin:${PATH}"

USER flytekit

# Install extensions using code-server
# Execution is performed here as code-server configuration depends on the USER setting
# If we install it as ROOT, the config will be stored in /root/.config/code-server/config.yaml
# Now, the config of code-server will be stored in /home/flytekit/.config/code-server/config.yaml
RUN code-server --install-extension /tmp/code-server/ms-python.python-2023.20.0.vsix \
    && code-server --install-extension /tmp/code-server/ms-toolsai.jupyter-2023.9.100.vsix

from flytekitplugins.flyin import (vscode, VscodeConfig, COPILOT_EXTENSION)
from flytekit import task, workflow
config = VscodeConfig()
config.add_extensions(COPILOT_EXTENSION)
@task
@vscode(
    config=config,
)
def t(a: int, b: int) -> int:
    return a + b

@workflow
def wf(a: int = 5, b: int = 3) -> int:
    out = t(a=a, b=b)
    return out

Screenshots

code-server --list-extensions

Test Screenshot

Check all the applicable boxes

• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.

Related PRs

Docs link

[codecov]

Codecov Report

Attention: 16 lines in your changes are missing coverage. Please review.

Comparison is base (ed91dd1) 85.73% compared to head (fa86816) 85.74%.
Report is 1 commits behind head on master.

Files	Patch %	Lines
...lyin/flytekitplugins/flyin/vscode_lib/decorator.py	38.46%	16 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1991      +/-   ##
==========================================
+ Coverage   85.73%   85.74%   +0.01%     
==========================================
  Files         313      313              
  Lines       23455    23492      +37     
  Branches     3516     3516              
==========================================
+ Hits        20109    20144      +35     
- Misses       2737     2738       +1     
- Partials      609      610       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[eapolinario]

now that we have more than one package in this plugin, can we use setuptools find_packages instead of listing them exhaustively here?

[Future-Outlier]

Yes, I can try it!

[Future-Outlier]

update: I will do this in a new PR.