Skip to content

Commit

Permalink
Handle null digest returned by batchStat
Browse files Browse the repository at this point in the history
When RemoteOutputService.BatchStat returns a null digest, the remote
output service knows the file exists, but it is incapable of returning
the digest. This may be caused by the kernel holding on to dirty pages
in its write-back cache.

By returning null, ActionMetadataHandler's
fileArtifactValueFromArtifact() will compute the file's digest
separately.

Make sure to run a recent build of bb-clientd to detect these cases.
  • Loading branch information
moroten committed Dec 21, 2023
1 parent a466dab commit 4388f9a
Showing 1 changed file with 20 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -398,7 +398,19 @@ public List<FileStatusWithDigest> batchStat(Iterable<PathFragment> paths)
}
FileStatus fileStatus = response.getFileStatus();
if (fileStatus.hasFile()) {
Digest digest = fileStatus.getFile().getDigest();
FileStatus.File regularFileStatus = fileStatus.getFile();
if (!regularFileStatus.hasDigest()) {
// The remote output service knows the file exists, but it
// is incapable of returning the digest. This may be caused
// by the kernel holding on to dirty pages in its write-back
// cache.
//
// By returning null, ActionMetadataHandler's
// fileArtifactValueFromArtifact() will compute the file's
// digest separately.
return null;
}
Digest digest = regularFileStatus.getDigest();
return new RegularFileStatus(digest.getSizeBytes(), DigestUtil.toBinaryDigest(digest));
}
if (fileStatus.hasDirectory()) {
Expand Down Expand Up @@ -558,6 +570,13 @@ protected byte[] getFastDigest(PathFragment path) throws IOException {
FileStatus fileStatus = response.getFileStatus();
if (fileStatus.hasFile()) {
FileStatus.File regularFileStatus = fileStatus.getFile();
if (!regularFileStatus.hasDigest()) {
// The remote output service knows the file exists, but it is
// incapable of returning the digest. This may be caused by
// the kernel holding on to dirty pages in its write-back
// cache. We must compute the digest ourselves.
return super.getFastDigest(path);
}
return DigestUtil.toBinaryDigest(regularFileStatus.getDigest());
}
if (fileStatus.hasExternal()) {
Expand Down

0 comments on commit 4388f9a

Please sign in to comment.