-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a new method to the PipelineResource interface for customization … #1215
Conversation
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: dlorenc The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
…of the Output directory We previously initialized each Output Resource with an empty directory. This changes the behavior to initialize this with a custom set of files per resource. The behavior now is the same as before, with each resource embedding a helper to create empty directories. Over time we can customize this where necessary.
As you know from #1188 I'm not a huge fan of struct embedding XD I have a couple other ideas but I'm okay with going ahead with this too! vs. #1188 I like that This is more of a long term thing and I think you were suggesting something like this @dlorenc , but And we could do it without struct embedding, dig this partial example: // Interface for PipelineResources
type PipelineResourceInterface interface {
// These are all the same
GetName() string
GetType() PipelineResourceType
Replacements() map[string]string
// The functionality we've been looking at is now input vs. output and `PodModifier` is the interface
GetInputStepModifer() PodModifier
GetOutputStepModifier() PodModifier
}
// This interface returns steps to add
type ResourceSteps interface {
Get(sourcePath string) ([]Step, error)
}
// This interface returns volumes to add
type ResourceVolumes interface {
Get(spec *TaskSpec) ([]corev1.Volume, error)
}
// Describes how to modify a Pod to implement an input or output resource
type PodModifier interface {
// For steps that have to go before the "actual steps" (for outputs too!) copied from the proposal in #238 :innocent:
GetBeforeSteps(name, sourcePath string) ResourceSteps
// For steps that go after the "actual steps"
GetAfterSteps(name, sourcePath string) ResourceSteps
// I am scared of the fact that a TaskSpec is getting passed in but that's another story :D
GetVolumes(spec *TaskSpec) ResourceVolumes
}
// Used when an empty dir needs to be created
type CreateEmptyDirSteps struct {}
func (CreateEmptyDirSteps*) Get(name, sourcePath string) ([]Step, error) {
return []Step{CreateDirStep(name, sourcePath)}, nil
}
// The pod modifier for image inputs
type ImageResourceInputPodModifier struct {}
func (ImageResourceInputPodModifier *) GetBeforeSteps(name, sourcePath string) ResourceSteps {
return CreateEmptyDirSteps{}
}
// ....
// NoResourceSteps just returns no steps
// Can be used by `NoPodModifications` but also can be returned by a PodModifier for any function that returns `ResourceSteps`
type NoResourceSteps {}
func (NoResourceSteps*) Get(sourcePath string) ([]Step, error) { return []Step{}, nil }
// NoResourceVolumes just returns no volumes
type NoResourceVolumes {}
func (NoResourceVolumes*) Get(spec *TaskSpec) ([]corev1.Volume, error) { return []corev1.Volume{}, nil}
// A PodModifier that does nothing
// For example GetOutputStepModifier could return this if outputs arent supported
type NoPodModifications struct { }
// Here is how PodModifier would indicate nothing needs to be done
func (NoPodModifications*) GetBeforeSteps(name, sourcePath string) ResourceSteps { return NoResourceSteps{} }
func (NoPodModifications*) GetAfterSteps(name, sourcePath string) ResourceSteps { return NoResourceSteps{} }
func (NoPodModifications*) GetVolumes(spec *TaskSpec) ResourceVolumes { return NoResourceVolumes{} } What I like about this is that most of the code that has to be written is going to be describing the functionality of the PipelineResources only, using (hopefully) small, reusable, clearly named and easily testable chunks of logic. It's interesting also b/c whatever we settle on for But after all that I'm happy to lgtm this as well and keep the discussion going! |
I like your proposal around the "GetInputModifier" and "GetOutputModifier"s! I think we'll have to consider the order in which they're applied, but I think it's pretty good overall and might actually make testing much easier. Let me give that one a try. I had a similar idea, but I was also trying to separate out Input and Output resources at the same time and that got too complicated. |
@dlorenc are you going to submit a new version of this? Shall we talk about this as part of the PipelineResources WG as well? |
Closing this for now. |
…of the Output directory
Changes
We previously initialized each Output Resource with an empty directory. This changes the
behavior to initialize this with a custom set of files per resource. The behavior now is
the same as before, with each resource embedding a helper to create empty directories.
Over time we can customize this where necessary.
Submitter Checklist
These are the criteria that every PR should meet, please check them off as you
review them:
Ref #1214