Concise Yaml for pipeline declaration.

[tejal29]

Expected Behavior

user should be able to write minimal lines of yaml to declare a pipeline.

Actual Behavior

we end up writing a number of redundant fields like "name" in inputSourceBindings might not be required.

Steps to Reproduce the Problem

1. Try writing a sample pipeline
2. Remove repeated fields.

Additional Info

[tejal29]

One way to address this is removing TaskRef and using a string instead.

[bobcatfish]

I took a quick look through our example yaml and these are the things I noticed that we might not need:

• Source Bindings do not need names
• I don't think Results need names:
  https://github.com/knative/build-pipeline/blob/86bcf4ecd93736cb8ad8413515ee3ff774d360cf/examples/pipelineparams.yaml#L15-L27

At the moment it also looks like params could be simpler - since they are currently just a name - but I think instead of simplifying this we should actually add more fields, see #190.

[abayer]

At @dlorenc's talk at Devops World/Jenkins World (augh awkward long conference names), and in discussions he and I had during the conference, he advocated for the idea that the Build Pipeline YAML is the "machine code" (for lack of a better term), which can be generated either via an editor or through transformation from another syntax. I'm intending down the road to add support for transforming Jenkins Declarative Pipeline syntax into Build Pipeline YAML, e.g.

I mention this because I think it is important to have the canonical representation be clearly unambiguous and useful for transformation from/to other forms, and for utilization in whatever implementation of Build Pipeline. That said, the particular cases cited above do seem to not create any ambiguity if they were removed, so hey. =)

[bobcatfish]

he advocated for the idea that the Build Pipeline YAML is the "machine code" (for lack of a better term), which can be generated either via an editor or through transformation from another syntax

That's an interesting point @abayer , so I think this would imply that the users of the YAML are the tools generating it, and the APIs consuming it, but not actually the end users of the Pipeline itself, is that accurate?

(+ @tejal29 )

[abayer]

Well, optionally - it may be better to think of it as end users may be writing the YAML directly, but an implementation/utilization of Build Pipeline may use a different syntax (like I mentioned, in my specific case, I'm thinking about migration paths for existing Jenkins users of its Declarative Pipeline syntax) that "transpiles" down to the standard YAML.

My thinking is heavily influenced by the idea of Build Pipeline as a spec as much as an implementation - I'm focused on the authoring experience (admittedly from the Jenkins perspective currently) and absolutely adore the idea of a standard definition of what a Pipeline can do that you can expect to have working on any implementation of the Build Pipeline "spec", with at the very least the same conceptual model and building blocks across the board.

[bobcatfish]

Note to self: Also need to update user study in @tejal29 's branch and decide where to put user study.

[tejal29]

@abayer i agree with your view that the yaml should be as unambiguous as it can and if it leads to very very long config, we can create tools for better user experience.
However, my initial concern is to make sure we don't repel users because of these long configs.
We need to make sure, we are not adding to the user pain.

[tejal29]

he advocated for the idea that the Build Pipeline YAML is the "machine code" (for lack of a better term), which can be generated either via an editor or through transformation from another syntax

That's an interesting point @abayer , so I think this would imply that the users of the YAML are the tools generating it, and the APIs consuming it, but not actually the end users of the Pipeline itself, is that accurate?

(+ @tejal29 )

yes, but we need to support both consumers (real users and tool) until we provide a tool which does that.

[dlorenc]

To clarify my thoughts a little further - we'll basically have two classes of users here (I think this is true for basically all Kubernetes yamls today): the users interfacing with Kubernetes directly and the users interfacing with higher level tooling built on top of Kubernetes.

In the long term we'll likely have way more of the second class of user - people using this through other systems. In the short term, we don't have any of those systems yet :)

I think we need to make sure the system is designed for both use-cases. YAML isn't a great user experience no matter how simple we make it, so we should expect and design for systems built on top. But that's not an excuse to make this overly verbose and complicated now, otherwise no one will be able to understand it enough to build something on top.