Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ContainerState and ContainerName for Sidecars #2075

Merged
merged 1 commit into from
Feb 25, 2020

Conversation

danielhelfand
Copy link
Member

@danielhelfand danielhelfand commented Feb 19, 2020

Closes #2074

Changes

Aligns the SidecarState with StepState to make the ContainerState and ContainerName available on TaskRun statuses for sidecars. I can add documentation as well under TaskRun Status if there would be a good place for it.

Submitter Checklist

Reviewer Notes

If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.

Release Notes

Add ContainerState and ContainerName to SidecarState to make sidecar statuses available on TaskRun status

@tekton-robot tekton-robot requested review from abayer and a user February 19, 2020 23:01
@googlebot googlebot added the cla: yes Trying to make the CLA bot happy with ppl from different companies work on one commit label Feb 19, 2020
@tekton-robot tekton-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Feb 19, 2020
@danielhelfand danielhelfand force-pushed the sidecar_containerstate branch 2 times, most recently from 5642b7b to 606340f Compare February 20, 2020 00:22
@tekton-robot tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 20, 2020
@bobcatfish
Copy link
Collaborator

This looks great @danielhelfand thanks for adding this!

I'm really glad to see the new unit tests, could we add another level of test as well? A couple options:

  • A test at the reconciler level, which behaves like a unit test but is in a lot of ways an integration test since it brings so many components together
  • An end to end test <-- i'm guessing (hoping?) there is an existing end to end side car test that could be updated to check for this status info

@danielhelfand
Copy link
Member Author

  • A test at the reconciler level, which behaves like a unit test but is in a lot of ways an integration test since it brings so many components together

Looking into TaskRuns at the reconciler level, I don't see any testing for sidecars, so this might be a good chance to add some testing there.

@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
test/builder/sidecar.go Do not exist 33.3%
test/builder/task.go 81.9% 81.5% -0.3

@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
test/builder/sidecar.go Do not exist 66.7%
test/builder/task.go 81.9% 81.5% -0.3

@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
test/builder/sidecar.go Do not exist 66.7%
test/builder/task.go 81.9% 81.5% -0.3

@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
test/builder/sidecar.go Do not exist 66.7%
test/builder/task.go 81.9% 81.5% -0.3

@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
test/builder/sidecar.go Do not exist 66.7%
test/builder/task.go 81.9% 81.5% -0.3

@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
test/builder/sidecar.go Do not exist 66.7%
test/builder/task.go 81.9% 81.5% -0.3

@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
test/builder/sidecar.go Do not exist 66.7%
test/builder/task.go 81.9% 81.5% -0.3

@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
test/builder/sidecar.go Do not exist 66.7%
test/builder/task.go 81.9% 81.5% -0.3

@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
test/builder/sidecar.go Do not exist 66.7%
test/builder/task.go 81.9% 81.5% -0.3

@danielhelfand
Copy link
Member Author

/hold

Unfortunately, there are issues related to #1347 with this approach. If a sidecar runs longer than all the Steps, the ContainerStatus is left as running for the SidecarStatus on the TaskRunStatus. I'll need to look into how to update the ContainerStatus for Sidecars in this situation.

@tekton-robot tekton-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 22, 2020
@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
test/builder/sidecar.go Do not exist 66.7%
test/builder/task.go 81.9% 81.5% -0.3

@danielhelfand
Copy link
Member Author

danielhelfand commented Feb 22, 2020

The integration test failures are all similar to as follows:

I0222 17:12:59.655]       imageID: docker-pullable://tianon/true@sha256:009cce421096698832595ce039aa13fa44327d96beedb84282a69d3dbcf5a81b
I0222 17:12:59.655]       name: slow-sidecar
I0222 17:12:59.655]       terminated:
I0222 17:12:59.655]         containerID: docker://2f3f434d4a4986faa4145af3434e694aa24a48b030f6e514da5e209648de027e
I0222 17:12:59.655]         exitCode: 127
I0222 17:12:59.655]         finishedAt: "2020-02-22T17:05:00Z"
I0222 17:12:59.656]         message: 'OCI runtime create failed: container_linux.go:345: starting container
I0222 17:12:59.656]           process caused "exec: \"sleep\": executable file not found in $PATH": unknown'
I0222 17:12:59.658]         reason: ContainerCannotRun
I0222 17:12:59.658]         startedAt: "2020-02-22T17:05:00Z"

My guess is that showing this ContainerStatus information now when a Sidecar has been stopped using the nop-image is showing the failure of trying to run sleep or any command not on the nop-image.

What could be done, in order to avoid this is, is to set a Terminated status for the SidecarStatus that explains the Sidecar was stopped successfully using the nop-image.


// update SidecarStatuses based on Pod ContainerStatuses when Sidecars
// have been stopped.
func updateSidecarStatus(pod *corev1.Pod, tr *v1alpha1.TaskRun, c *Reconciler) error {
Copy link
Member Author

@danielhelfand danielhelfand Feb 22, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of using current State (s.State) from ContainerStatus on Pod with stopped Sidecars, what could be done is to create a Terminated Status with the following information:

corev1.ContainerState{
    Terminated: &corev1.ContainerStateTerminated{
        ExitCode: 0,
	Reason:   "Completed",
        Message:  "Sidecar container successfully stopped by nop-image",
        StartedAt: <Use value from LastTerminationState instead of current State (s.LastTerminatedState.Terminated.StartedAt)>,
        FinishedAt: <Use value from LastTerminationState instead of current State (s.LastTerminatedState.Terminated.FinishedAt)>,
        ContainerID: <Use value from LastTerminationState instead of current State (s.LastTerminatedState.Terminated.ContainerID)>,
}

Using the LastTerminationStatus, if exit code == 137, it can be assumed that the sidecar was killed.

Using the LastTerminationStatus would also provide more accurate information about the Start/End time of the Sidecar as well as information about the actual image that was used for the Sidecar.

@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
test/builder/sidecar.go Do not exist 66.7%
test/builder/task.go 81.9% 81.5% -0.3

@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
test/builder/sidecar.go Do not exist 66.7%
test/builder/task.go 81.9% 81.5% -0.3

@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
test/builder/sidecar.go Do not exist 66.7%
test/builder/task.go 81.9% 81.5% -0.3

@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
test/builder/sidecar.go Do not exist 66.7%
test/builder/task.go 81.9% 81.5% -0.3

@danielhelfand
Copy link
Member Author

/hold cancel

Should be ready for another look

@tekton-robot tekton-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 23, 2020
Copy link
Member

@vdemeester vdemeester left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me
/cc @chmouel

@tekton-robot tekton-robot requested a review from chmouel February 24, 2020 12:21
Copy link
Member

@vdemeester vdemeester left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Feb 25, 2020
@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sbwsg, vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot tekton-robot merged commit aecf943 into tektoncd:master Feb 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cla: yes Trying to make the CLA bot happy with ppl from different companies work on one commit lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add ContainerState to SidecarState
5 participants