Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a new method to the PipelineResource interface for customization … #1215

Closed
wants to merge 1 commit into from

Conversation

dlorenc
Copy link
Contributor

@dlorenc dlorenc commented Aug 19, 2019

…of the Output directory

Changes

We previously initialized each Output Resource with an empty directory. This changes the
behavior to initialize this with a custom set of files per resource. The behavior now is
the same as before, with each resource embedding a helper to create empty directories.

Over time we can customize this where necessary.

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

Ref #1214

@googlebot googlebot added the cla: yes Trying to make the CLA bot happy with ppl from different companies work on one commit label Aug 19, 2019
@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dlorenc

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot tekton-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 19, 2019
…of the Output directory

We previously initialized each Output Resource with an empty directory. This changes the
behavior to initialize this with a custom set of files per resource. The behavior now is
the same as before, with each resource embedding a helper to create empty directories.

Over time we can customize this where necessary.
@bobcatfish
Copy link
Collaborator

As you know from #1188 I'm not a huge fan of struct embedding XD I have a couple other ideas but I'm okay with going ahead with this too! vs. #1188 I like that EmptyDirOutputResource has a clear name and responsibility.

This is more of a long term thing and I think you were suggesting something like this @dlorenc , but
Looking at the fact that we've got GetUploadSteps and GetOutputInitializationSteps now, maybe we can split up the input/output functionality?

And we could do it without struct embedding, dig this partial example:

// Interface for PipelineResources
type PipelineResourceInterface interface {
       // These are all the same
	GetName() string
	GetType() PipelineResourceType
	Replacements() map[string]string

        // The functionality we've been looking at is now input vs. output and `PodModifier` is the interface
        GetInputStepModifer() PodModifier
        GetOutputStepModifier() PodModifier
}

// This interface returns steps to add
type ResourceSteps interface {
     Get(sourcePath string) ([]Step, error)
}

// This interface returns volumes to add
type ResourceVolumes interface {
    Get(spec *TaskSpec) ([]corev1.Volume, error)
}

// Describes how to modify a Pod to implement an input or output resource 
type PodModifier interface {
       // For steps that have to go before the "actual steps" (for outputs too!) copied from the proposal in #238 :innocent: 
	GetBeforeSteps(name, sourcePath string) ResourceSteps
        // For steps that go after the "actual steps"
  	GetAfterSteps(name, sourcePath string) ResourceSteps
        // I am scared of the fact that a TaskSpec is getting passed in but that's another story :D
        GetVolumes(spec *TaskSpec) ResourceVolumes
}

// Used when an empty dir needs to be created
type CreateEmptyDirSteps struct {}
func (CreateEmptyDirSteps*) Get(name, sourcePath string) ([]Step, error) {
    return []Step{CreateDirStep(name, sourcePath)}, nil
}

// The pod modifier for image inputs
type ImageResourceInputPodModifier struct {}
func (ImageResourceInputPodModifier *) GetBeforeSteps(name, sourcePath string) ResourceSteps {
    return CreateEmptyDirSteps{}
}
// ....

// NoResourceSteps just returns no steps
// Can be used by `NoPodModifications` but also can be returned by a PodModifier for any function that returns `ResourceSteps`
type NoResourceSteps {}
func (NoResourceSteps*) Get(sourcePath string) ([]Step, error) { return []Step{}, nil }

// NoResourceVolumes just returns no volumes
type NoResourceVolumes {}
func (NoResourceVolumes*) Get(spec *TaskSpec) ([]corev1.Volume, error) { return []corev1.Volume{}, nil}

// A PodModifier that does nothing
// For example GetOutputStepModifier could return this if outputs arent supported
type NoPodModifications struct { }

// Here is how PodModifier would indicate nothing needs to be done
func (NoPodModifications*) GetBeforeSteps(name, sourcePath string) ResourceSteps { return NoResourceSteps{} }
func (NoPodModifications*) GetAfterSteps(name, sourcePath string) ResourceSteps { return NoResourceSteps{} }
func (NoPodModifications*) GetVolumes(spec *TaskSpec)  ResourceVolumes { return NoResourceVolumes{} }

What I like about this is that most of the code that has to be written is going to be describing the functionality of the PipelineResources only, using (hopefully) small, reusable, clearly named and easily testable chunks of logic.

It's interesting also b/c whatever we settle on for PipelineResourceInterface is pretty much gonna turn into the interface folks have to implement for extensible resource (#238 ) .

But after all that I'm happy to lgtm this as well and keep the discussion going!

@dlorenc
Copy link
Contributor Author

dlorenc commented Aug 21, 2019

I like your proposal around the "GetInputModifier" and "GetOutputModifier"s! I think we'll have to consider the order in which they're applied, but I think it's pretty good overall and might actually make testing much easier.

Let me give that one a try. I had a similar idea, but I was also trying to separate out Input and Output resources at the same time and that got too complicated.

@afrittoli
Copy link
Member

@dlorenc are you going to submit a new version of this? Shall we talk about this as part of the PipelineResources WG as well?

@dlorenc
Copy link
Contributor Author

dlorenc commented Sep 23, 2019

Closing this for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cla: yes Trying to make the CLA bot happy with ppl from different companies work on one commit size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants