If you want to build multi-architecture Docker containers in GitHub Actions, the standard recommendation you’ll find online is to install BuildX and QEMU. The downside of this approach is that QEMU emulation is about 10x slower than native hardware. Building my simple Hello World project went from 30 seconds to 3 minutes.
In this post, I will show you several ways to speed up your builds. The options you have are:
- Switching to runners that have cross-platform remote builds pre-configured
- Building single-architecture images in a matrix and merging them manually
- Using GitHub Actions instances as remote builders with Tailscale
- Setting up your own machines for remote builds
- Using a Kubernetes cluster for remote builds
Switching to a different Runner Provider
This is the simplest solution, but it requires signing up for a new service and will cost some money. There are alternative runner providers that have their runners preconfigured with BuildX and remote builders with native hardware.
One such provider that I tried is Namespace.so.
Signing up was fast and easy, and switching to them in my workflows
only required changing the runs-on field in my YAML.
Building Single Images and Merging
This option is probably the simplest way to get cross-platform builds without leaving GitHub. You build an individual image for each of the architectures that you want, then merge them with a buildx command like:
|
|
In this example,
I use a matrix of jobs to reduce duplicate YAML, and then a merge job to create the final image.
GitHub Remote Builders with Tailscale
Honestly, this is cool in a nerdy way,
but I wouldn’t recommend it for production.
For each hardware architecture, I spin up a job that starts a buildkitd server.
I then join my tailnet with a pre-determined hostname that allows the builder to find the machine.
The final step in the job is printf "HTTP/1.1 200 OK\r\nContent-Length: 16\r\n\r\nShutting down..." | nc -l -p 8080 which just waits until someone hits port 8080 and then shuts down.
The main build step configures itself with the remote builders from the previous step.
It then joins the tailnet and uses those remote instances to do a cross-platform build.
After the build is done, I use a curl command to cause the other jobs to end.
Like I said, this is a pretty cool setup, but there’s just so much that can go wrong.
If the curl fails, you’ll get jobs that hang for a long time,
and you’ll have to worry about tailnet configurations and security.
Kubernetes Remote Builders
If you happen to have a Kubernetes cluster that has both Intel and ARM nodes in it, you can use them as remote builders. In this example, I create a temporary namespace for each build, run the builds there, and then clean up afterwards.
Overall this is not a bad option if you already have a Kubernetes cluster being used for other purposes. But you probably don’t want to be creating one just for the purpose of your builds.
TCP Remote Builders
You can also just run individual VMs of your different hardware types and use them for remote builds. In this example, I created one ARM and one Intel VM and secured them with TLS certs. I then configured BuildX to use those remote runners for the build. You could also leverage Tailscale for this and avoid the need for TLS certificates.
Conclusion
So there you have it, several options to get faster multi-architecture builds on GitHub Actions. Personally, I currently lean towards Namespace.so simply because it only costs me about $2 a month and I’m lazy. If Namespace started to get expensive, I would probably go with the separate builds and merge pattern. And if my builds were starting to get expensive on GitHub, I might look into setting up builders at home and doing remote builds over Tailscale.