-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[TEP-0135] Coschedule per (Isolated) PipelineRun e2e support #6927
[TEP-0135] Coschedule per (Isolated) PipelineRun e2e support #6927
Conversation
Skipping CI for Draft Pull Request. |
The following is the coverage report on the affected files.
|
The following is the coverage report on the affected files.
|
a4315b0
to
200b7b0
Compare
The following is the coverage report on the affected files.
|
200b7b0
to
255f1e5
Compare
The following is the coverage report on the affected files.
|
The following is the coverage report on the affected files.
|
} | ||
} | ||
case aa.AffinityAssistantPerPipelineRun, aa.AffinityAssistantPerPipelineRunWithIsolation: | ||
affinityAssistantName := GetAffinityAssistantName("", pr.Name) | ||
if err := c.KubeClientSet.AppsV1().StatefulSets(pr.Namespace).Delete(ctx, affinityAssistantName, metav1.DeleteOptions{}); err != nil && !apierrors.IsNotFound(err) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we also need to delete the PVCs created by the statefulsets, and update integration tests to ensure those PVCs are actually deleted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@@ -159,12 +161,16 @@ func TestCreateAndDeleteOfAffinityAssistantPerPipelineRun(t *testing.T) { | |||
}, { | |||
name: "other Workspace type", | |||
pr: testPRWithEmptyDir, | |||
expectStatefulSetSpec: nil, | |||
expectStatefulSetSpec: &appsv1.StatefulSetSpec{}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: It doesn't really make sense to differentiate between nil and a pointer to an empty struct; I'd have tests treat them as equivalent rather than asserting a specific behavior.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Prior to this commit, we don't create AffinityAssistant
for TaskRuns
without a pvc
based workspace in coschedule per pipelinerun
mode, the statefulset
is NOT expected to be created (nil
).
In this commit, we create AffinityAssistant
for all TaskRuns
, so we do expect an AffinityAssitant
is created (but there is no pvc
or volumeclaimtemplate
in the StatefulSetSpec
)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm a bit confused, why would we want to validate that a statefulset is created with an empty spec? how is that different from a nil pointer?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for the confusion. The confusing point is that we had a StatefulSetSpec
filter here:
if d := cmp.Diff(expectStatefulSetSpec, &aa.Spec, podSpecFilter, podTemplateSpecFilter); d != "" { |
Replica
and Selector
before.
In the test case, it is actually a spec with Replica
and Selector
but not an empty spec (we didn't have it before since they will not be validated anyways due to the filter). I think the best way address this confusion is to remote this filter and validate the selector and replicas (and we get more validated coverage 😄 )
} | ||
} | ||
|
||
func resetFeatureFlagAndCleanup(ctx context.Context, t *testing.T, c *clients, namespace string) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is interesting-- how do you prevent the integration test from interfering with the other integration tests? Do they just run sequentially?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I'm following the discussion here: #6079 to set/revert the feature flags and run tests sequentially.
Thanks @QuanZhang-William! One more note; I think the existing release note might be hard for users to understand, and they might not know why it might be a good idea to enable this new feature. Can you put the release notes in terms of the functionality the user is interested in, i.e. scheduling pods to the right nodes? Also, are all the docs for this feature up to date? I wouldn't want to have a release note saying the feature is ready but have our docs say it's not functional yet. |
/hold until #6929 is merged so that this PR can have e2e support for both |
255f1e5
to
1318420
Compare
The following is the coverage report on the affected files.
|
No problem, Lee. I have updated the release not
We have this open PR #6892 for the documentation. I will remove all the WIP and Warnings in the documentation PR after merging this one. |
t.Errorf("expected err type mismatching, expecting %v but got: %v", ErrAffinityAssistantCreationFailed, err) | ||
} | ||
} | ||
if d := cmp.Diff(tc.expectedErr.Error(), err.Error()); d != "" { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you can do this with cmpopts.EquateErrors I think? This will check that the error is the right type which is more important than validating the exact error message
@@ -159,12 +161,16 @@ func TestCreateAndDeleteOfAffinityAssistantPerPipelineRun(t *testing.T) { | |||
}, { | |||
name: "other Workspace type", | |||
pr: testPRWithEmptyDir, | |||
expectStatefulSetSpec: nil, | |||
expectStatefulSetSpec: &appsv1.StatefulSetSpec{}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm a bit confused, why would we want to validate that a statefulset is created with an empty spec? how is that different from a nil pointer?
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: lbernick The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
e0d1168
to
53e71bf
Compare
The following is the coverage report on the affected files.
|
The following is the coverage report on the affected files.
|
/assign |
/hold until v0.50 LTS is released |
53e71bf
to
c8a8c7f
Compare
/assign @JeromeJu |
The following is the coverage report on the affected files.
|
The following is the coverage report on the affected files.
|
Part of [tektoncd#6740]. [TEP-0135][tep-0135] introduces a feature that allows a cluster operator to ensure that all of a PipelineRun's pods are scheduled to the same node. This commit consumes the functions added in [tektoncd#6819] to implement end to end support of `Coschedule:PipelineRuns` where all the `PipelineRun pods` are scheduled to the same node, and the `Coschedule:isolate-pipelinerun` coschedule modes where only 1 PipelineRun is allowed to run in a node at the same time. /kind feature [tektoncd#6819]: tektoncd#6819 [tektoncd#6740]: tektoncd#6740 [tep-0135]: https://github.com/tektoncd/community/blob/main/teps/0135-coscheduling-pipelinerun-pods.md
c8a8c7f
to
224cba3
Compare
/hold cancel |
The following is the coverage report on the affected files.
|
The following is the coverage report on the affected files.
|
/lgtm |
Changes
Part of #6740. TEP-0135 introduces a feature that allows a cluster operator to ensure that all of a PipelineRun's pods are scheduled to the same node.
This commit consumes the functions added in #6819 and implements end to end support of
coschedule:pipelineruns
coscheduling mode, where all thePipelineRun pods
are scheduled to the same node./kind feature
Submitter Checklist
As the author of this PR, please check off the items in this checklist:
/kind <type>
. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tepRelease Notes