Support pipeline matrix feature

[afrittoli]

Is your feature request related to a problem ?

This is related to a new pipeline feature - tekton matrix.
Matrix requires support for minimal status, which already implemented.
Dashboard today however only displays the first TaskRun in a matrix:

While in fact there are many TaskRuns

Describe the solution you'd like

I think a basic feature would be to show all TaskRun in the PipelineRun view.
It would be nice to have the TaskRun that belong to a matrix visually grouped together somehow.
Additionally it would be nice to highlight the parameters that are passed to the specific each TaskRun that come from the matrix.

[AlanGreene]

Thanks for the feature request @afrittoli and for adding the link in #2306.

I think we'll have to tackle this one in at least 2 phases:

1. show all related TaskRuns in the PipelineRun view
2. improve the display so that all TaskRuns for the same matrix are visually grouped or otherwise connected

I'm hoping that 1 should be relatively straightforward to implement but will need to experiment with the feature to ensure we have all of the required information to reliably generate a stable UI + URL (e.g. there's a number appended to the pipelinetask name, need to ensure that this number or the expected display name is available in a label or other known field so we have a stable source for this info).

For 2, this will require some more work and will be tracked as part of the larger redesign effort in #2306. We already have a similar issue with retries where each attempt is rendered in the UI as if it was an independent TaskRun but we want to group them in some way, ideally showing a single TaskRun with some option to view details of each attempt. For matrix we may end up with a different presentation but to avoid too much difference in both design and implementation we should at least consider these two features together.

[AlanGreene]

Some very rough testing for the first phase using the example at https://github.com/tektoncd/pipeline/blob/main/examples/v1beta1/pipelineruns/alpha/pipelinerun-with-matrix.yaml

Before:

After:

There are some issues rendering retries in the Dashboard with this update, and also possibly a bug in Pipelines where the PipelineRun is terminated without waiting for retries to complete (will be tracked separately).

This POC also relies on some potentially brittle logic to get a unique display name for each TaskRun in the matrix, needs more testing, particularly with longer PipelineRun and pipelineTask names. Having the generated name <pipelineTask>-<index> (e.g. platforms-and-browsers-0, platforms-and-browsers-1, etc.) available as a label on the TaskRun may help clean some of this up, further exploration needed to determine if there's a reliable way to achieve this purely in the Dashboard code.

We should also look at how the CLI is currently handling this and any planned changes / improvements there to ensure at least some sort of consistency: https://github.com/tektoncd/cli/issues/1661

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[AlanGreene]

This is something we still want to do, freezing so it doesn't get auto-closed.
/lifecycle frozen

[AlanGreene]

/area roadmap

[ileixe]

From user perspective, This is very critical issue to use matrix feature because there's no way to see failed logs in dashboard at all currently.

+1

[AlanGreene]

Hi @ileixe, thanks for your interest in this feature and apologies for the delay, I didn't see the notification for your message. I agree that it's important to have this functionality, and there is some work in progress to add matrix support. However, bear in mind that matrix is currently an alpha feature in Tekton Pipelines and subject to change.

I have been doing some work on matrix support in the Dashboard this week. At this point I'm not sure which release it will be included in, either:

• a very basic version in the upcoming v0.39.0 release due in the next few days, or
• in a nightly release next week, then included in the v0.40.0 release next month

This plan is of course subject to change depending on changes in the functionality provided by Tekton Pipelines, results of testing, etc.

Until matrix support is provided natively in the Dashboard you can view the logs by accessing the TaskRun details page for the relevant run(s). If you need to filter the TaskRuns list to find the relevant runs you can filter by PipelineRun name tekton.dev/pipelineRun:<pipelineRunName> and/or pipeline task name tekton.dev/pipelineTask:<pipelineTaskName>

See for example on our dogfooding cluster, to get the TaskRuns for the 'unit-tests' tasks in the specified PipelineRun:

[AlanGreene]

Initial support for displaying matrix TaskRuns was released in Dashboard v0.39.0

Future releases will continue to improve on this.

[AlanGreene]

Some related proposals in Tekton Pipelines that will allow us to further improve the display of matrix TaskRuns in the Dashboard:

• TEP-0150 displayName in matrix community#1122
  • implementing TEP-0150 - displayName in childReferences pipeline#7683
• Detect when a taskRun is part of matrix  pipeline#7564
• matrix instance count in taskRun pipeline#7562

[AlanGreene]

Matrix display names are now supported in Dashboard v0.45.0, this feature requires Pipelines v0.58.0 or newer: https://github.com/tektoncd/dashboard/releases/tag/v0.45.0