Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[KEP-0009] feat: add expression based assertions #576

Open
wants to merge 28 commits into
base: main
Choose a base branch
from

Conversation

kumar-mallikarjuna
Copy link
Contributor

What this PR does / why we need it:
This PR adds CEL-expression based assertions to TestAsserts. See https://github.com/kudobuilder/kuttl/blob/main/keps/0009-expression-based-assertions.md for more details.

Fixes #562

Copy link
Member

@porridge porridge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, but needs a few changes.

pkg/apis/testharness/v1beta1/test_types.go Outdated Show resolved Hide resolved
pkg/apis/testharness/v1beta1/test_types.go Outdated Show resolved Hide resolved
pkg/test/utils/kubernetes.go Outdated Show resolved Hide resolved
pkg/test/utils/kubernetes.go Outdated Show resolved Hide resolved
pkg/test/utils/kubernetes.go Outdated Show resolved Hide resolved
variables[resourceRef.Id] = referencedResource.Object
}

env, err := cel.NewEnv()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Calling cel.NewEnv() may not be enough, you probably want to enable a couple of options/libs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eddycharly any hints on what might be useful? 🤔

@kumar-mallikarjuna kumar-mallikarjuna force-pushed the KEP-0009 branch 2 times, most recently from 0fe748e to a85752a Compare December 3, 2024 07:56
pkg/apis/testharness/v1beta1/expression.go Outdated Show resolved Hide resolved
pkg/apis/testharness/v1beta1/expression.go Outdated Show resolved Hide resolved
pkg/apis/testharness/v1beta1/expression.go Outdated Show resolved Hide resolved
pkg/test/step.go Outdated Show resolved Hide resolved
pkg/test/utils/kubernetes.go Outdated Show resolved Hide resolved
pkg/test/utils/kubernetes.go Outdated Show resolved Hide resolved
pkg/test/utils/expression.go Outdated Show resolved Hide resolved
pkg/test/utils/kubernetes.go Outdated Show resolved Hide resolved
pkg/test/utils/kubernetes.go Outdated Show resolved Hide resolved
@kumar-mallikarjuna
Copy link
Contributor Author

Thanks for the quick review @porridge . I'll add the tests tomorrow.

Copy link
Member

@porridge porridge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks much better, but some more changes are needed. 🙏🏻
Please see inline.

pkg/test/step.go Outdated Show resolved Hide resolved
pkg/test/expression_integration_test.go Show resolved Hide resolved
pkg/expressions/cel.go Outdated Show resolved Hide resolved
pkg/expressions/cel.go Outdated Show resolved Hide resolved
pkg/expressions/cel.go Outdated Show resolved Hide resolved
pkg/test/step.go Outdated Show resolved Hide resolved
pkg/apis/testharness/v1beta1/expression.go Outdated Show resolved Hide resolved
pkg/expressions/cel.go Outdated Show resolved Hide resolved
pkg/expressions/cel.go Show resolved Hide resolved
pkg/test/expression_integration_test.go Outdated Show resolved Hide resolved
This PR adds CEL-expression based assertions to `TestAsserts`. See https://github.com/kudobuilder/kuttl/blob/main/keps/0009-expression-based-assertions.md for more details.

Signed-off-by: Kumar Mallikarjuna <[email protected]>
Signed-off-by: Kumar Mallikarjuna <[email protected]>
Signed-off-by: Kumar Mallikarjuna <[email protected]>
Signed-off-by: Kumar Mallikarjuna <[email protected]>
Signed-off-by: Kumar Mallikarjuna <[email protected]>
Signed-off-by: Kumar Mallikarjuna <[email protected]>
Signed-off-by: Kumar Mallikarjuna <[email protected]>
Copy link
Member

@porridge porridge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost there!
Just a bunch of nitpicks for messages and identifier names, and one issue in the test..

pkg/expressions/cel.go Outdated Show resolved Hide resolved
pkg/expressions/cel.go Outdated Show resolved Hide resolved
pkg/expressions/cel.go Outdated Show resolved Hide resolved
pkg/expressions/cel.go Outdated Show resolved Hide resolved
pkg/expressions/cel.go Outdated Show resolved Hide resolved
pkg/test/expression_integration_test.go Outdated Show resolved Hide resolved
pkg/test/expression_integration_test.go Outdated Show resolved Hide resolved
variables[resourceRef.Id] = referencedResource.Object
}

env, err := cel.NewEnv()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eddycharly any hints on what might be useful? 🤔

Signed-off-by: Kumar Mallikarjuna <[email protected]>
Signed-off-by: Kumar Mallikarjuna <[email protected]>
Signed-off-by: Kumar Mallikarjuna <[email protected]>
Copy link
Member

@porridge porridge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more things.

pkg/test/expression_integration_test.go Show resolved Hide resolved
pkg/expressions/cel.go Outdated Show resolved Hide resolved
pkg/expressions/cel.go Outdated Show resolved Hide resolved
pkg/expressions/cel.go Outdated Show resolved Hide resolved

for i := 0; i < len(files)-1; i++ {
fName := fmt.Sprintf("%s/%s", dirName, files[i].Name())
step := buildTestStep(t)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only now I realized there is a separate step object for every file, and that this logic makes certain expectations about directory content. Also the directories look like kuttl tests but the test semantics are somewhat different.

Since the invalid expression case contains just one file, and all cases are effectively a single step, how about:

  • using a single step object per test,
  • with a bunch of LoadYAML calls, each using the same expectLoadFailure check
  • and with a single step.Run call
  • and drop the numeric prefix from filenames to avoid implying that there can be multiple steps.

Copy link
Contributor Author

@kumar-mallikarjuna kumar-mallikarjuna Jan 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and all cases are effectively a single step

check_expression_for_ephemeral_namespace has two steps so I wanted to keep an inclusive logic 🤔

pkg/test/expression_integration_test.go Outdated Show resolved Hide resolved
Signed-off-by: Kumar Mallikarjuna <[email protected]>
Signed-off-by: Kumar Mallikarjuna <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add CEL expression support
3 participants