Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Preserve analysis cache through --test_env changes #24316

Closed
wants to merge 2 commits into from
Closed

Preserve analysis cache through --test_env changes #24316

wants to merge 2 commits into from

Conversation

fmeum
Copy link
Collaborator

@fmeum fmeum commented Nov 13, 2024 •
edited
Loading

Fixes #7450

RELNOTES[INC]: Changing --test_env no longer invalidates the analysis cache. ctx.configuration.test_env may be empty for non-test rules and should not be used by such rules.

@fmeum fmeum marked this pull request as ready for review November 13, 2024 20:45
@fmeum fmeum requested a review from a team as a code owner November 13, 2024 20:45
@fmeum fmeum requested review from katre and removed request for a team November 13, 2024 20:45
@github-actions github-actions bot added team-Configurability platforms, toolchains, cquery, select(), config transitions awaiting-review PR is awaiting review from an assigned reviewer labels Nov 13, 2024
@katre
Copy link
Member

katre commented Nov 13, 2024

Will mark as ready for merge (or merge myself) once the tests are fixed.

@fmeum fmeum requested a review from a team as a code owner November 13, 2024 22:06
fmeum
fmeum commented Nov 13, 2024
View reviewed changes
Copy link
Collaborator Author

@fmeum fmeum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed the failing test.

src/main/java/com/google/devtools/build/lib/runtime/CommandEnvironment.java
@@ -329,7 +330,7 @@ public void exit(AbruptExitException exception) {
}
}
for (Map.Entry<String, String> entry :
options.getOptions(CoreOptions.class).testEnvironment) {
options.getOptions(TestOptions.class).testEnvironment) {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@katre I'm not 100% sure whether this will work as is - would build and info need to declare that they use TestOptions?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just add a check that TestOptions is present: if it's absent the entire block can be skipped.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm worried that it just won't ever be there for info. The tests didn't trigger any crashes though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you wrap the entire for loop in if (options.contains(TestOptions.class)) then it'll be safe for any command.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will be safe in the sense of "doesn't crash", but will this still be a backwards compatible change in the sense that bazel info client-env shows test env entries?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I was able to run a quick demo:

$ FOO=bar ~/repos/bazel/bazel-bin/src/bazel --host_jvm_debug info --test_env=FOO client-env
Running host JVM under debugger (listening on TCP port 5005).
build --test_env=FOO=bar

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. In that case I would leave it as is so that we get a crash instead of silent loss of functionality if this ever breaks. What do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM.

@katre katre added awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally and removed awaiting-review PR is awaiting review from an assigned reviewer labels Nov 14, 2024
@github-actions github-actions bot removed the awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally label Nov 15, 2024
@fmeum fmeum deleted the 7450-test-env branch November 15, 2024 23:30
@fmeum
Copy link
Collaborator Author

fmeum commented Nov 15, 2024

@iancha1992 Could you mark this for 8.1.0?

@meteorcloudy
Copy link
Member

This was rolled-back at 1da4641

@meteorcloudy
Copy link
Member

@katre Can you please check if what's the reason of the internal breakage?

@katre
Copy link
Member

katre commented Nov 18, 2024

I'll investigate.

@katre
Copy link
Member

katre commented Nov 18, 2024

Okay, it turns out that there are exactly three config_setting targets in the google depot that check on the value of test_env, and when that is removed due to test configuration trimming, it causes an error.

I'm going to look into whether it's possible to fix these.

@fmeum
Copy link
Collaborator Author

fmeum commented Nov 18, 2024

Would it be possible to mark them testonly? Is Google using the experimental flag that disables trimming for testonly targets?

@katre
Copy link
Member

katre commented Nov 18, 2024

Using testonly is my plan, but clearly there are some non-test dependencies, so I'll need to find and clean those up as well. I'll check the testonly flag and see what the status is.

@katre
Copy link
Member

katre commented Nov 18, 2024

Can you rebase this on to HEAD and I'll try again?

@fmeum
Copy link
Collaborator Author

fmeum commented Nov 19, 2024

Sent #24398

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@katre katre katre approved these changes

Assignees
No one assigned
Labels
team-Configurability platforms, toolchains, cquery, select(), config transitions
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

--test_env (and some other test flags) should not discard analysis cache with --trim_test_configuration
3 participants
@fmeum @katre @meteorcloudy