Docker Compose

Automate docker compose files update

In this scenario, we have to maintain a docker-compose file and we want to be sure that we always use the newest docker images available.

Docker image versions can be identified using two methods, either we use the docker image tag or the docker image digest.

Tag

A docker image tag is useful when we want to have an easy to identify human-readable version like in nginx:1.17 where 1.17 is the tag, unfortunately, a docker image tag can be easily overridden, on purpose or not. Like when using the tag latest. This means that a docker image tag can’t be considered as a source of truth.

Digest

The second method to identify a docker image version is by using its digest like in nginx@sha256:8269a7352a7dad1f8b3dc83284f195bac72027dd50279422d363d49311ab7d9b where 8269a7352a7dad1f8b3dc83284f195bac72027dd50279422d363d49311ab7d9b is the digest. A digest is a hash referencing a specific version of a docker image tag like in this case nginx:1.17. A digest is immutable, we can always rollblack to a specific digest if need. But we must admit that it’s not convenient to identify which version we are referencing to without having to dig into it.

The problem here could be resumed by "Do we want to only use well-defined docker image by using digests but with the cost of losing readability?" or "Are we ok to take the risk of running an unwanted version by using image tags?".

It depends on how important we value readability versus uniqueness, but a rule of thumb is to consider the docker image build strategy, what’s the risk of running an unwanted version and how big is the chance of having to rollback. If we only build one image by version and that version is regularly updated then preferring readability by using docker image tag seems ok-ish otherwise in any other scenarios It’s usually safer to use docker image digest.

Now that we know what we need to update and why, let see how we can do it using updatecli.

Docker Image Tag

When we want to automate image tag updates, we have to identify the process used to define those tags, and if that process is automated, then you can more than likely also automate version update that were updatecli can play a role.

In updatecli, you specify this process in the source stage, then if needed you apply some condition and finally if everything goes well then you update the version in your target file. So let take this example where we want to regularly update the Jenkins version, we know that a weekly release is published every week on a maven repository, then a docker image build and published on DockerHub. Considering the build strategy, it seems fine to use docker image tag.

Let’s consider this docker compose file:

docker-compose.yaml
version: '3'
services:
  jenkins:
    image: jenkins/jenkins:2.245
    ports:
        - 8080:8080

Now we write an updatecli configuration that contains a source, a conditions, and one target

strategy.yaml
sources:
  jenkins:
    kind: maven
    postfix: "-jdk11" # will be add to the value returned by the spec definition
    transformers:
      - addSuffix: "v"
    spec:
      owner: "maven"
      url: "repo.jenkins-ci.org"
      repository: "releases"
      groupID: "org.jenkins-ci.main"
      artifactID: "jenkins-war"
conditions:
  docker:
    name: "Docker Image Published on Registry"
    kind: dockerImage
    spec:
      image: "jenkins/jenkins"
targets:
  imageTag:
    name: "docker-compose jenkins service"
    kind: "yaml"
    prefix: "jenkins/jenkins:" # will be add to the value returned by the spec definition
    spec:
      file: "./docker-compose.yaml"
      key: "services.jenkins.image"

docker run -i -t -v $(pwd):/updatecli --workdir /updatecli ghcr.io/updatecli/updatecli:v0.6.1 diff --config ./strategy.yaml

Updatecli will query the maven repository 'releases' located on repo.jenkins-ci.org, to have the latest version. If it finds one, then we add to the value retrieved, the prefix jenkins/jenkins: and the suffix -jdk11 as we are looking specifically for the java 11 version. We validate that docker image effectively exist on DockerHub and then we test if the value in the file 'docker-compose.yaml' for the key services.jenkins.image is correct otherwise we bump it.

If we are good with the changes then we apply them by using apply instead of diff.

docker run -i -t -v $(pwd):/updatecli --workdir /updatecli ghcr.io/updatecli/updatecli:v0.6.1 apply --config ./strategy.yaml

Docker Image Digest

As explained earlier, sometimes we prefer to use docker image digest, if we need to rely on a specific version or if we want to easily rollback. While most of the configuration remains the same as in the previous example, this time the source will be dockerhub and we won’t test if an image already exists on DockerHub as we already ask DockerHub for the latest image available.

docker-compose.yaml
version: '3'
services:
  jenkins:
    image: jenkins/jenkins:2.245
    ports:
        - 8080:8080
strategy.yaml
sources:
  jenkins:
    kind: dockerDigest
    spec:
      image: "jenkins/jenkins"
      tag: "lts-jdk11"
targets:
  imageTag:
    name: "jenkins/jenkins:lts-jdk11 docker digest"
    kind: yaml
    spec:
      file: "./docker-compose.yaml"
      key: "services.jenkins.image"

This time updatecli queries DockerHub to retrieve the digest for the docker image jenkins/jenkins:lts-jdk11. If it finds one, then we test if the value in the file 'docker-compose.yaml' for the key services.jenkins.image is correct otherwise we change it.

Again if we are ok with the changes then we apply them by using apply instead of diff.

  • docker run -i -t -v $(pwd):/updatecli -workdir /updatecli ghcr.io/updatecli/updatecli:v0.6.1 diff --config strategy.yaml

  • docker run -i -t -v $(pwd):/updatecli -workdir /updatecli ghcr.io/updatecli/updatecli:v0.6.1 apply --config stragegy.yaml

Git/GitHub

Now that we have an easy way to update docker image version, we are missing a way to save, review, rollback those changes, and git for this is a tremendous tool. Either we directly commit and push to a git repository or we use the GitHub workflow by pushing to a temporary branch. If we are using GitHub we can also we submit our changes via a pull request which can then be approved.

docker-compose.yaml
version: '3'
services:
  jenkins:
    image: jenkins/jenkins:2.245
    ports:
        - 8080:8080

While the configuration remains quite similar to our previous example, this time we introduce two new elements. First, strategy.yaml becomes strategy.tpl which is a go template. By using go template we can define generic values and reference them from our template or read values from an environment variable like {{ requiredEnv GITHUB_TOKEN }}. The second major change is the 'scm' block which should be quiet obvious and defines where to push commits.

strategy.tpl
sources:
  jenkins:
    kind: dockerDigest
    spec:
      image: "jenkins/jenkins"
      tag: "lts-jdk11"
targets:
  imageTag:
    name: "jenkins/jenkins:lts-jdk11 docker digest"
    kind: yaml
    spec:
      file: "./docker-compose.yaml"
      key: "services.jenkins.image"
    scm:
      github:
        user: "John"
        email: "john@example.com"
        owner: "jenkins-infra"
        repository: "charts"
        token: "{{ requiredEnv GITHUB_TOKEN }}"
        username: "johnDoe"
        branch: "main"

And now you can use the same command than before

  • docker run -i -t -v $(pwd):/updatecli -workdir /updatecli ghcr.io/updatecli/updatecli:v0.6.1 diff --config.tpl

  • docker run -i -t -v $(pwd):/updatecli -workdir /updatecli ghcr.io/updatecli/updatecli:v0.6.1 apply --config.tpl

Conclusion

In this scenario, we saw how to automatically update docker-compose file using custom strategies by using updatecli. Updatecli is a small tool that can be used from your favorite CI environment.

Now we can replace our docker-compose file by any other YAML file to automate YAML update.

Edit this page on GitHub