[Core feature] Allow to specify userdata bucket in the storage configuration in flyte-core charts

[gdabisias]

Motivation: Why do you think this is important?

Currently it is only possible to specify the metadata container in the storage object in the flyte core chart, e.g.

storage: {
			type: "custom"
			custom: {
				type: "stow"
				stow: {
					kind: "s3"
					config: {
						auth_type:     "accesskey"
						region:        "xxx"
						access_key_id: string | *""
						secret_key:    string | *""
						endpoint:      xxx
						disable_ssl:   false
						v2_signing:    false
					}
				}
            container: helm.values.userSettings.metadataBucket

while in the single binary deployment it is possible to specify both user and metadata containers in the storage configuration, e.g.

 storage:
    # metadataContainer Bucket to store Flyte metadata
    metadataContainer: "my-organization-flyte-container"
    # userDataContainer Bucket to store Flyte user data
    userDataContainer: "my-organization-flyte-container"

from here

The current fix for this lack of specification is to define the following field in the propeller configuration

core: propeller: {
				"rawoutput-prefix":     "s3://" + helm.values.userSettings.userdataBucket + "/"
...

Goal: What should the final outcome look like, ideally?

We should use the same storage configuration for the flyte deployment charts.

I think it would be good for 2 reasons:

• it unifies configuration across deployments
• it allows the users to specify all storage configurations just in one place

Describe alternatives you've considered

None

Propose: Link/Inline OR Additional context

No response

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[eapolinario]

@gdabisias , would you be willing to contribute with this change?

[gdabisias]

hi @eapolinario, possibly in the future, but not right now:)