CertGrid CertGrid
Hands-on Lab·Docker

Docker Build Cache in CI/CD

A CI worker starts with an empty cache every run, which is why naive pipelines rebuild everything every time. Push the cache to a registry instead - proven here by a brand new builder reporting CACHED on a layer it never built.

Operations and Troubleshooting Guide 36 of 46 Advanced

Tested on the versions above. The workflow file is shown as a file. It was not executed on GitHub - the Docker commands it runs were executed locally, and those transcripts are the ones shown.

One Docker host is all this guide needs. Nothing here depends on a second machine, and the hardware above is modest on purpose - a 2 core, 4 GB VM runs everything in this path.
Server NameIP AddressOSRolesCPURAMHDD
DOCKER01192.168.0.21Ubuntu 26.04 LTSDocker Host2 Core4 GB50 GB

Before you start

  1. The problem CI has that your laptop does not

    Your machine keeps its build cache between builds, so the second build of a project is fast. A CI runner is a fresh machine every time: no images, no layers, no cache. Unless you give the cache somewhere to live between runs, every pipeline pays full price for every layer, including the dependency install that never changes.

    bash Example session 
    docker buildx versiongithub.com/docker/buildx v0.36.1 1d8dde89b8aba914e05e45366770736fea1fd690# buildx can export the cache to a registry, which is what makes it survive the runner

    Expected resultA buildx version. Anything from v0.10 supports the cache exports used here.

    Success conditionYou have buildx. The default docker driver cannot export cache - you need a docker-container builder, created below.

  2. Create a builder that can export cache

    The docker-container driver runs BuildKit in its own container, which is what unlocks registry cache export and multi-platform builds. Note network=host here: without it the builder cannot reach a registry on the host's localhost, because the builder container has its own.

    bash Example session 
    docker buildx create --name cg-builder --driver docker-container --driver-opt network=host --bootstrap#1 creating container buildx_buildkit_cg-builder0 0.4s done#1 DONE 9.7scg-builder

    Expected resultThe builder container created and bootstrapped.

    Success conditionThe builder exists. In GitHub Actions the docker/setup-buildx-action step does exactly this for you.

  3. Push the cache to a registry

    --cache-to type=registry writes the cache as an ordinary image tag alongside your build; --cache-from reads it back. mode=max exports intermediate layers too, not just the final ones - more upload, far better hit rate on the next run.

    bash Example session 
    docker buildx build --builder cg-builder -t localhost:5000/cg-ci:2 --cache-to type=registry,ref=localhost:5000/cg-ci:cache,mode=max --cache-from type=registry,ref=localhost:5000/cg-ci:cache --push .#7 DONE 0.2s > importing cache manifest from localhost:5000/cg-ci:cache

    Expected resultA cache tag sitting in the registry next to your image tags.

    Verify it worked

    bash 
    curl -s http://localhost:5000/v2/cg-ci/tags/list{"name":"cg-ci","tags":["1","2","3","cache"]}

    Success conditionThe cache is a registry artefact now. It outlives the machine that produced it, which is the entire point.

  4. Prove it works on a machine that has never built this

    This is the test that matters. Delete the builder entirely and make a new one - a fresh BuildKit instance with an empty local cache, standing in for tomorrow's CI runner. It reports CACHED for the package install it has never run, because it pulled that layer from the registry.

    bash 
    docker buildx rm cg-builder && docker buildx create --name cg-builder2 --driver docker-container --driver-opt network=host --bootstrapcg-builder2cg-builder removed#1 [internal] booting buildkit#1 pulling image moby/buildkit:buildx-stable-1#1 pulling image moby/buildkit:buildx-stable-1 21.3s done#1 creating container buildx_buildkit_cg-builder20#1 creating container buildx_buildkit_cg-builder20 0.4s done#1 DONE 21.7sdocker buildx build --builder cg-builder2 -t localhost:5000/cg-ci:3 --cache-from type=registry,ref=localhost:5000/cg-ci:cache --push .#6 [2/2] RUN apk add --no-cache curl#6 CACHED

    Expected resultCACHED on a builder created seconds ago.

    Success conditionA brand new builder skipped the install. That is the saving your pipeline gets on every run after the first.

  5. The workflow file

    This is the whole thing in GitHub Actions form. Three official actions do the setup, and the build step carries the cache flags you just used by hand. gha cache is the GitHub-hosted alternative to a registry cache and needs no registry credentials, but it is scoped to the repository and has its own size limits.

    bash 
    cat .github/workflows/docker.ymlname: docker on:  push:    branches: [main] jobs:  build:    runs-on: ubuntu-latest    steps:      - uses: actions/checkout@v4      - uses: docker/setup-buildx-action@v3      - uses: docker/login-action@v3        with:          username: ${{ secrets.DOCKERHUB_USERNAME }}          password: ${{ secrets.DOCKERHUB_TOKEN }}      - uses: docker/build-push-action@v6        with:          push: true          tags: myorg/myapp:latest          cache-from: type=gha          cache-to: type=gha,mode=max

    Expected resultA workflow with checkout, buildx setup, registry login and a cached build.

    Success conditionYou can map every line to something you ran by hand above. Nothing here is magic - the actions are wrappers over the same commands.

  6. Tag with the commit, deploy by digest

    Tagging with github.sha gives every build a unique, traceable name - never rely on latest from CI. The digest is stronger still: it is the content address of exactly that build, so deploying by digest removes any possibility of the tag having moved underneath you.

    bash Example session 
    docker buildx imagetools inspect localhost:5000/cg-ci:1 --format "{{ .Manifest.Digest }}"sha256:2dd2a150c294f6984e676418b7230fe26b7b1c3aaa7845c362c1179fe7bcfe1d

    Expected resultA sha256 digest for the pushed image.

    Success conditionYou have an immutable reference. build-push-action exposes this as an output, so a deploy job downstream can consume it directly.

  7. Clean up

    Builders are containers and they persist until removed, holding their cache with them. On a long-lived runner this is a real disk consumer.

    bash 
    docker buildx rm cg-builder2docker rm -f cg-registry# check for the anonymous volume the registry leaves behind: docker volume ls

    Expected resultThe builder and registry removed.

    Success conditiondocker buildx ls no longer lists your builder. Remember the registry container leaves an anonymous volume - remove it too.

Troubleshooting

Official sources

← All Docker guides