The example configuration for
bb-deployments has been updated.
This post will give an overview of what has happened since the last update
summary in November 2023, and will cover
important changes since then up to 2026-06-12.
Additional Buildbarn changes can be found in the bb-deployments
changelog.
Major changes
Upgrade to Bazel 9
The Buildbarn components have been upgraded to use Bazel 9.
Buildbarn binaries will now relaunch themselves in a child process, if the PID
is 1. This removes the need to run bb_runner through tini because zombie
processes in containers are no longer an issue.
The gRPC servers can now forward incoming streams of specified services. This
enables passing Bazel's build event stream to the same DNS name, without having
to add an extra L7 router in front of the bb_storage frontend.
grpcServers: [{
...
relays: [
{
endpoint: {
address: 'localhost:1234',
},
services: [
'build.bazel.remote.execution.v2.Execution',
'com.google.devtools.build.v1.PublishBuildEvent',
],
},
],
}],
Buildbarn's HTTP client now supports OAuth Client Credential flow.
An implementation of WinFSP VFS has been added, enabling bb_worker to
succesfully execute Bazel builds with virtualised file inputs on Windows. See
the commit message for more details.
Buildbarn has expanded the configuration of JMESPath expressions. Instead of
only specifying a string, an optional set of files can be included, whose
contents will be available for the expression using the files field. The
contents of the file refreshes every 60 seconds. The struct also includes
optional test vectors, each test consisting of an input and expected output. The
program fails to start if the expression does not produce the expected output
from the input.
addMetadataJmespathExpression: {
expression: |||
{
"authorization": [std.format('bearer %s', files.token)]
}
|||,
files: [
{
key: "token",
path: "/tokens/buildbarn",
},
],
},
ZSTD compression
Support has been added for in-transit ZSTD compression in the gRPC ByteStream
layer
(Jul 15, 2025), used by the CAS. This feature is a pre-requisite for supporting
compressed bb_clientd. A pool of decoders and encoders can be configured with
an optional upper bound of
instances
(Mar 12, 2026). See this commit for the gRPC client side
configuration
(Mar 2, 2026).
The previous sharding algorithm has been replaced with Rendezvous hashing,
making resharding less disruptive. This change is not backwards compatible. See
Buildbarn ADR
#11
for more details.
HTTP and gRPC servers can now forward authentication and authorization requests
to a remote service. The results are cached for a short while to reduce the load
on the remote service.
NFSv4 changes
NFSv4.1 for bb_worker is now
supported
(Jul 9, 2024) in addition to NFSv4.0. The versions have some substantial
protocol differences, as described in the commit message.
NFSv4 has been extended to support named attributes, see the related
PR (Oct 5, 2025)
for more details.
Predeclared platform queue size classes are now defined by an array of size
classes, previously configured by setting the maximum size class. This allows
all workers for all sizes to be autoscaled down to zero. When using multiple
size classes, the lowest cannot be zero.
{
...
predeclaredPlatformQueues: [
{
instanceNamePrefix: 'testingQueue',
sizeClasses: [1, 2, 3, 4, 5, 6, 7],
},
{
sizeClasses: [1, 2, 3, 4, 5, 6, 7],
},
],
...
}
// If the user does not care about multiple size classes:
{
...
predeclaredPlatformQueues: [
{
sizeClasses: [0],
},
],
}
macOS mount configuration changes
The support for FUSE mounts on macOS has been
removed
(Mar 3, 2024) due to OSXFUSE/macFUSE being unstable. macOS users are recommended
to instead use NFSv4. However the NFSv4 support for macOS 14 and older has been
dropped
(May 7, 2026) as macOS 15 has been out for some time.
REv2 has added the ability to upload output directories as multiple Directory
messages instead of a
single Tree object, if requested by the client. bb_worker has added support for
this feature, and can also force upload Directory messages to the CAS in
addition to uploading the Tree object by setting
forceUploadTreesAndDirectories: true;
in the worker configuration. Forcefully uploading Directory messages has several
advantages as documented in the worker Proto
file:
-
bb_browser is capable of displaying listings of individual directories
contained in an output directory without needing to load the full Tree object
from the CAS.
-
Root directories of output directories created through bb_clientd's Bazel
Output Service feature load slightly faster, as they can be validated without
processing full Tree objects.
-
Even for clients that only support output directories in the form of Tree
messages, having Directory messages present means that subsequent build
actions need to upload fewer objects.
The disadvantage of enabling this option is that a larger number of objects
are written into the CAS.
Minor changes
Extended digest algorithm support
The list of supported digest algorithms has been extended with
blake3
(Dec 31, 2025) and
GITSHA1
(Jan 27, 2026)
This allows the scheduler to delegate action routing decisions to a remote gRPC
service.
The gRPC server component now supports basic ATLS authentication.
Additional OIDC support
The authenticator has been extended to support
PKCE
(Oct 31, 2025), and user information claims can now be extracted from an ID
token
(May 28, 2025) instead of the user info endpoint.
A configuration option has been added which allows bb_worker to control of the
execution timeout timer via HTTP. It is configured by specifying two URLs which
are polled by the worker: a "suspend URL", whose response controls whether the
timeout timer is suspended, and a "resume URL" whose response controls whether
to resume the timer. This feature is useful for when the worker timeout needs to
be compensated for file downloads from outside the input root.
Buildbarn now includes simple program which saves JWKSes in a Kubernetes
ConfigMap, which can then be made available to Buildbarn with a volume mount.
The program is meant to be run periodically in the cluster.
The scheduler can now be configured with an additional authorizer, to authorize
bb_worker synchronize requests.
VFS and FUSE backends now support setting directory ownership. This is useful
for when running repo rules as remote actions, as
Bonanza will, many of which depend on
having ownership.
A decorator, DeadlineEnforcingBlobAccess, has been added which sets a
configurable timeout duration. This sets an upper limit on the durations for
bb_storage's incoming RPCs. For example, this is useful for reject writes that
are taking an excessive amouint of time to complete.
The default gRPC connection load balancing policy is now configurable. This
enables using the round-robin strategy. See gRPC service
configuration for more
information.
The recommended values from keyLocationMapMaximumGetAttempts and
keyLocationMapMaximumPutAttempts have been doubled. For more information about
the Key Location Map, see our blog post about
the topic.
The HTTP server configuration now supports an optional TLS configuration. This
is useful when running Buildbarn without an ingress controller, for example in a
bare metal environment.
URI subject alternative names can now be matched with JMESPath expressions when
validating TLS certificates.
Custom Kubernetes resolvers can be configured in the global configuration. The
URL schema and Kubernetes API server is specified, whose endpoints are expanded
and registered. This is useful in a bare-metal cluster with Buildbarn pods using
the host network, where it can be difficult to establish network connections
between components.
Before uploading the output files, bb_worker will now wait for them to be
closed, with a user-specified upper bound for the waiting duration. This
prevents cases where the output files are still open for writing when they are
uploaded, due to the kernel closing files asynchronously.
Every action now gets its own "server_logs" directory, created by bb_worker.
bb_runner doesn't do anything with the directory, but a custom runner can for
example use the directory to store core dumps of failed actions, which bb_worker
will then include in the upload. For more information on how to configure the
directory, see the commit
message.
A global option has been added which calls setrlimit(2) with the configured
values as arguments. The keys in the configuration correspond to the RLIMIT_*
suffixes in the setrlimit resource parameter.
global: {
...
setResourceLimits: {
"NOFILE": {
softLimit: 1024,
hardLimit: 4096,
},
"CPU": {
softLimit: 2,
hardLimit: 4,
},
},
...
}
This feature was introduced because a new version of Go changed the logic for
setting RLIMIT_NOFILE as part of child
processes, risking build
actions to run into file descriptor limits. This change can help avoid this by
overriding the resource limits.
ReferenceExpandingBlobAccess has been extended to support request forwarding
to a REv2 CAS. This is useful when designing asset storage services that provide
direct exposition over bytestream://.
Additional JWT support
JWKs can now optionally be read from a
file
(Oct 30, 2023), rather than than providing it inline in the configuration.
JWTs can now be signed with the Ed25519
algorithm
(Jun 19, 2024), and JWTs signed with RSA-PSS signatures can be
validated
(May 7, 2026).
