Skip to content

Commit

Permalink
Document the worker protocol in the example
Browse files Browse the repository at this point in the history
  • Loading branch information
aherrmann committed Oct 7, 2024
1 parent bbf8a85 commit f8c20ef
Showing 1 changed file with 38 additions and 0 deletions.
38 changes: 38 additions & 0 deletions examples/persistent_worker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,3 +99,41 @@ stderr for root//:demo-7 (demo):
Bazel persistent worker ...
...
```

## Protocol

### Starlark

A Buck2 persistent worker is created by a rule that emits the
`WorkerInfo` provider. Setting `remote = True` on this provider
indicates that this worker is remote execution capable.

Buck2 actions indicate that they can utilize a persistent worker by
setting the `exe` parameter to `ctx.actions.run` to
`WorkerRunInfo(worker, exe)`, where `worker` is a `WorkerInfo` provider,
and `exe` defines the fallback executable for non persistent-worker
execution.

Buck2 actions that want to utilize a remote persistent worker must pass
command-line arguments in an argument file specified as `@argfile`,
`-flagfile=argfile`, or `--flagfile=argfile` on the command-line.

### Local Persistent Worker

A locally executed Buck2 persistent worker falls under the [Buck2
persistent worker protocol](./proto/buck2/worker.proto): It is started
and managed by Buck2 and passed a file path in the `WORKER_SOCKET`
environment variable where it should create a gRPC Unix domain socket to
serve worker requests over. Multiple requests may be sent in parallel
and expected to be served at the same time depending on the
`concurrency` attribute of the `WorkerInfo` provider.

### Remote Persistent Worker

A remotely executed Buck2 persistent worker falls under the [Bazel
persistent worker protocol](./proto/bazel/worker_protocol.proto): It is
started and managed by the remote execution system. Work requests are
sent as length prefixed protobuf objects to the standard input of the
worker process. Work responses are expected as length prefixed protobuf
objects on the standard output of the worker process. The worker process
may not use standard output for anything else.

0 comments on commit f8c20ef

Please sign in to comment.