RemoteExecutionService: Action.Command to set output_paths

[sluongng]

This is a follow-up to #18198

Make Bazel compatible with newer version of Remote Api by setting
output_paths along-side output_directories and output_files. The latter
2 are deprecated in newer REAPI specification.

[sluongng]

cc: @fmeum @EdSchouten @tjgq

[tjgq]

How much of a concern is it that this change increases the Action.Command message size for every action?

An alternative would be to use the high_api_version reported in the ServerCapabilities message [1] to decide which field to use. But I'm not sure if the complication is worth it.

[1] https://cs.opensource.google/bazel/bazel/+/master:third_party/remoteapis/build/bazel/remote/execution/v2/remote_execution.proto;l=1799;drc=d0cba5507fcb5d636b1a9a3b1f58cf63314781c0

[sluongng]

How much of a concern is it that this change increases the Action.Command message size for every action?

An alternative would be to use the high_api_version reported in the ServerCapabilities message [1] to decide which field to use. But I'm not sure if the complication is worth it.

I have to scratch my head a bit on this.
Regarding Remote Cache / RBE implementation, most of the weight is on the CAS traffic and not on Action Cache(AC) side.
Increasing the Command message should have minimal impact.

The hard question would be when/how we deprecate the usages of the old fields.
That's when I think checking ServerSide capability would come into play:

1. In a major LTS release (i.e. 7.0.0), create a transition period when we only set the old fields when detecting the older version of the API.
2. In another major LTS release, remove the old field completely.

But I suspect there isn't much appetite for doing this deprecation.
Regardless, it's outside the scope of this PR.

[tjgq]

How much of a concern is it that this change increases the Action.Command message size for every action?

An alternative would be to use the high_api_version reported in the ServerCapabilities message [1] to decide which field to use. But I'm not sure if the complication is worth it.

I have to scratch my head a bit on this. Regarding Remote Cache / RBE implementation, most of the weight is on the CAS traffic and not on Action Cache(AC) side. Increasing the Command message should have minimal impact.

Fair enough, I'm happy to approve as-is. We can improve it later if any of the API implementors express concerns.

The hard question would be when/how we deprecate the usages of the old fields. That's when I think checking ServerSide capability would come into play:

1. In a major LTS release (i.e. 7.0.0), create a transition period when we only set the old fields when detecting the older version of the API.
2. In another major LTS release, remove the old field completely.

But I suspect there isn't much appetite for doing this deprecation. Regardless, it's outside the scope of this PR.

I'd expect some future Bazel version to start requiring at least v2.1 of the API when negotiating the server capabilities (currently, it requires at least v2.0). At that point we could stop sending the deprecated fields. I agree that this is future work.

[sluongng]

There is a CI failure on CentOS which I think was due to infra underneath not executing //src/test/java/com/google/devtools/build/lib/remote:remote fast enough leading to a timeout.

I suspect splitting the target into 5 different shards might help, but I will leave that to the maintainers to decide.
The exact target passed on the other linux platforms on CI.

[sluongng]

@bazel-io flag

[iancha1992]

@bazel-io fork 6.3.0