Inconsistent behavior after removing run_after_success

[IronPan]

What happened:
Here are some recent changes
#10694 (comment)
and #11046

The behavior is now different from what in the past.

• tests are run in sequence in stead of in parallel. We are experiencing long waiting time.
• In our case, we would like PRs to be blocked on build-image and e2e-test but not sample test, since that usually take long to run. This is not possible anymore since we only show single presubmit test in PR.

What you expected to happen:
Tests used to be in "run after" section can run in parallel.
We can take action based on individual test suit.

How to reproduce it (as minimally and precisely as possible):

Please provide links to example occurrences, if any:
https://gubernator.k8s.io/pr/kubeflow/pipelines/769

Anything else we need to know?:

[IronPan]

/assign @krzyzacy @fejta

[krzyzacy]

@fejta do we support parallel steps somehow?

@IronPan or we can make each e2e test do a image build and split them up again?

[fejta]

tests are run in sequence

Knative-build steps run in sequence. If you want things to be parallel you need two jobs and/or a build-pipeline.

Note that build-pipeline support doesn't exist right now.

In our case, we would like PRs to be blocked on build-image and e2e-test but not sample test

The two options here are
a) make sample test always exit 0, regardless of whether or not it produced test failures
b) Make sample tests its own top-level job (repeating the build step, if necessary)

[IronPan]

Thanks in that case we might just split test suit and build images individually.

related: tektoncd/pipeline#168