Add GCS Support

[matthewphsmith]

This adds support for uploading and downloading data to/from GCS.

[honnix]

@matthewphsmith What is the status of this PR? Would you like us to review as well? Thanks.

[matthewphsmith]

It's good to go from my perspective, I just need an approval on a review.

[wild-endeavor]

+1 bump version plz

[honnix]

s/s3/gs/g

[tnsetting]

I tried to the PR and the pyflyte-execute could download the input.pb from gcs and upload the output.pb to gcs. But the node still failed because of OutputsNotFoundError: Outputs not found for task type python-task.... It looks like the whole engine_dir_xxx folder was uploaded instead of just upload the .pb files.

[tnsetting]

I think we can change local_path to local_path + '/*' to just avoid upload the whole engine_dir folder.

[honnix]

The current behaviour in this PR seems fine to me, because the expectation of this function is to upload the specified local path. Maybe it is the caller who should use ../* instead?

[honnix]

We can confirm that gsutil and aws s3 behave differently when copying a dir recursively.

e.g:

$ tree test
test
└── test1
    └── test.txt

1 directories, 1 file

$ gsutil cp -r test/* gs://flyte-test
Copying file://test/test1/a/b/test.txt [Content-Type=text/plain]...
/ [1 files][    0.0 B/    0.0 B]
Operation completed over 1 objects.

$ gsutil ls gs://flyte-test
gs://flyte-test//
gs://flyte-test/test1/    <--------

vs.

$ gsutil cp -r test gs://flyte-test
Copying file://test/test1/a/b/test.txt [Content-Type=text/plain]...
/ [1 files][    0.0 B/    0.0 B]
Operation completed over 1 objects.

$ gsutil ls gs://flyte-test
gs://flyte-test//
gs://flyte-test/test/    <--------

[honnix]

I think there is a caveat using gsutil: it doesn't seem to respect GOOGLE_APPLICATION_CREDENTIALS environment variable. That means the container should either have a service account activated so it becomes ADC effectively, or carry an identity such as GCE.

[honnix]

Rebased and addressed the comments #41