Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve handling of compile_data with mixed sources #3176

Merged

Conversation

martingms
Copy link
Contributor

I'm not very well versed in starlark nor the rules_rust codebase, so feel free to ignore this and address the issue in a more fitting way, but this fixes #3171 and a related issue for me.

Copy link
Contributor

@sam-mccall sam-mccall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was looking at generated sources recently, this change makes sense to me.

Needs an owner, @krasimirgg we were talking about this very issue a couple of days ago :-)


# Optionally join compile data
if crate.compile_data:
compile_data = depset(ctx.files.compile_data, transitive = [crate.compile_data])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to worry about compile_data being None?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

afaict this shouldn't be possible, attr.label_list defaults to [], and we set compile_data explicitly to a depset based on that in all codepaths, but I can add this check back if you think it's better to be defensive. It was a bit confusing to me, which is why I removed it!

@martingms martingms force-pushed the compile_data-with-generated-srcs branch from 03dacc4 to 7c80de4 Compare January 13, 2025 09:32
@krasimirgg krasimirgg self-requested a review January 13, 2025 09:40
@krasimirgg
Copy link
Collaborator

Looks good! Please re-format the file that the Buildifier workflow is complaining about.

Copy link
Collaborator

@krasimirgg krasimirgg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fixes issue where generated compile_data didn't work in some cases.
@martingms martingms force-pushed the compile_data-with-generated-srcs branch from 46726d1 to e6f947e Compare January 13, 2025 09:45
@martingms
Copy link
Contributor Author

Sorry, missed buildifier and one test affected by the last commit, pushed an updated patch now 👍

@krasimirgg krasimirgg enabled auto-merge January 13, 2025 10:14
@krasimirgg krasimirgg added this pull request to the merge queue Jan 13, 2025
Merged via the queue into bazelbuild:main with commit bb74a65 Jan 13, 2025
3 checks passed
@martingms martingms deleted the compile_data-with-generated-srcs branch January 13, 2025 11:33
@UebelAndre
Copy link
Collaborator

UebelAndre commented Jan 21, 2025

This has lead to a regression for targets which have compile data from completely other packages. See repro at (#3193)

ERROR: /Users/user/Code/rules_rust/test/unit/compile_data/src/BUILD.bazel:11:13: in rust_library rule //test/unit/compile_data/src:lib:
Traceback (most recent call last):
	File "/Users/user/Code/rules_rust/rust/private/rust.bzl", line 87, column 32, in _rust_library_impl
		return _rust_library_common(ctx, "rlib")
	File "/Users/user/Code/rules_rust/rust/private/rust.bzl", line 152, column 55, in _rust_library_common
		srcs, compile_data, crate_root = transform_sources(ctx, ctx.files.srcs, ctx.files.compile_data, crate_root)
	File "/Users/user/Code/rules_rust/rust/private/utils.bzl", line 825, column 64, in transform_sources
		generated_compile_data = [_symlink_for_non_generated_source(ctx, src, package_root) for src in compile_data]
	File "/Users/user/Code/rules_rust/rust/private/utils.bzl", line 870, column 64, in _symlink_for_non_generated_source
		src_symlink = ctx.actions.declare_file(paths.relativize(src_short_path, package_root))
	File "/private/var/tmp/_bazel_user/76282c66b0dfe3c5cb9a230bdc913a52/external/bazel_skylib~/lib/paths.bzl", line 247, column 17, in _relativize
		fail("Path '%s' is not beneath '%s'" % (path, start))
Error in fail: Path 'test/unit/compile_data/data/data.txt' is not beneath 'test/unit/compile_data/src'
ERROR: /Users/user/Code/rules_rust/test/unit/compile_data/src/BUILD.bazel:11:13: Analysis of target '//test/unit/compile_data/src:lib' failed

I don't quite know how to solve for this as I'm not sure what the symlink path should be for this foreign path.

@krasimirgg
Copy link
Collaborator

Happy if we rolled this back until we have a full fix for this.

@martingms
Copy link
Contributor Author

martingms commented Jan 22, 2025

@krasimirgg The crash should be fixed by #3196 or similar, but it's unclear to me how the repro in #3193 could have worked (as in the data.txt file is actually available during compile) even before this (#3176) PR was merged, as generated sources were built in the out tree even before. I'm probably missing something! Do you have any pointers to what the situation is in the real world failures?

Fine with me to roll it back too obviously.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

compile_data does not work together with generated sources
4 participants