Skip to main content

BuildBar at the Meroton Office

· One min read
Benjamin Ingberg
Benjamin Ingberg

After BazelCon we've all gotten a bit giddy and you might feel excited how the information presented at BazelCon might impact your development workflow. For that purpose we're inviting everyone in the wider bazel community to an open BuildBar at the Meroton offices.

Come over and digest BazelCon with high level technical discussions of the talks, great company, a pleasant atmosphere and also beer.

Feel welcome to come over this Thursday the first of December from 16 to 20.

Directions

You'll find us at our Linköping offices at Fridtunagatan 33. Currently there is some construction but follow the red lines and you'll be fine.

Fridtunagatan 33 Linköping

Remote Executors for the Free Environment

· 2 min read
Benjamin Ingberg
Benjamin Ingberg

We've performed some updates to the free tier and our pricing model.

Shared Cache

The free tier has been upgraded into an environment with a 1TB cache. This means that you can use the free tier without worrying about hitting any limits or setup process.

Do note that while the cached items will only be accessible with the correct api keys, the storage area can be reclaimed by anyone. I.e. as the cache becomes full your items might be dropped.

With the current churn we expect any items in the cache last at least a week. However, you should always treat items in the cache as something which might be dropped at any moment. The purpose and design of the cache is to maximize build performance, not to provide storage.

Shared executors

New for the free environment is the introduction of remote execution, this was previously only available for paying customers.

If you're using bazel you can simply add the --remote_executor=<...> flag and your builds will be done remotely.

The free environment has access to 64 shared executors.

Starter tier

If you need dedicated storage for evaluation purposes we also offer a starter tier. The starter tier is a streamlined environment with 100 GB of dedicated cache and access to the same 64 executors as the free tier.

This allows you to try out remote execution and caching without being disturbed by others. While the 100GB dedicated cache will only be used by you and therefore not be overwritten by anyone else's build, the cache is still a cache and the system will still drop the cache entries if it determines it is useful for maximizing performance.

Need more executors?

At the moment the starter tier has a fixed amount of shared executors. If you need a scalable solution contact us for setting up a custom Buildbarn environment of any size.

Tips, Tricks & Non-Deterministic Builds

· 2 min read
Benjamin Ingberg
Benjamin Ingberg

When you have a remote build and cache cluster it can sometimes be hard to track down what exactly is using all of your building resources. To help with this we have started a tips and trix section in the documentation where we will share methods we use to debug and resolve slow builds.

The first section is about build non-determinism. Ideally your build actions should produce the same output when run with the same input, in practice this is sometimes not the case. If you are lucky a non-deterministic action won't be noticed since the inputs for the non-deterministic action is unchanged it won't be rebuilt.

If you're not so lucky the non-determinism stems from a bug in the implementation and you should definitely pay attention to them. But how do you know which if any actions are non-deterministic?

This is not trivial but we have added a server side feature which allows detection of non-determinism with virtually no effort on your part.

Once activated it reruns a configured fraction of your actions and automatically flags them if they produce different outputs. The scheduling is done outside of your bazel invocation so your build throughput will be unaffected at the cost of an increase in the number of resources consumed. We suggest 1% which will only increase your resource use by a trivial amount but you could of course set it to 100% which would double the cost of your builds.

Updates to Buildbarn deployment repo as of April 2022

· 2 min read
Benjamin Ingberg
Benjamin Ingberg

The sample configuration project for Buildbarn was recently updated after a long hiatus. As an aid for people to understand which changes have been done see the following high level summary.

April 2022 Updates

This includes updates to Buildbarn since December 2020.

Authorizer Overhaul

Authorizers have been rehauled to be more flexible it is now part of each individual cache and execution configuration.

Using a JWT authorization bearer token has been added as an authorization method.

Hierarchical Blob Access

Using hierarchical blob access allows blobs in instance name foo/bar to be accessed from instance foo/bar/baz but not instance foo or foo/qux.

Action Result Expiration

An expiry can be added to action result which lets the action cache purge the result of an exection that was performed too far in the past. This can be used to ensure that all targets are rebuilt periodically even if they are accessed frequently enough to not normally be purged from the cache.

Read Only Cache Replicas

Cache read traffic can now be sent to a read-only replica which is periodically probed for availability.

Concurrency Limiting Blob Replication

Limit the number of concurrent replications to prevent network starvation

Run Commands as Another User

Allows the commands to be run as a different user, on most platforms this means the bb-runner instance must run as root.

Size Class Analysis

Allows executors of different size classes to be used, the scheduler will attempt to utilize executors efficiently but there is an inherent tradeof between throughput and latency. Once configured the scheduler will automatically attempt to keep track of which actions are best run on which executors.

Execution Routing Policy

The scheduler accepts an execution routing policy configuration that allows it to determine how to defer builds to different executors.

If you see any other changes you feel should get a mention feel free to submit a pull request at github using the link below.

Purpose of the Articles

· One min read
Benjamin Ingberg
Benjamin Ingberg

The purpose of these articles is to have a freeform area discussing ideas, technical issues, solutions and news in an indepth relaxed manner. It is not to serve as reference material, structured reference material should be available in the documentation section.

The article format allows more in depth on discussions for reacurring subjects. In contrast to the documentation published articles aren't changed, if the subject requires a revisit in the future then we publish a new post and add references to the old.

If you see any errors feel free to submit a pull request at github using the link below.