Guide: docker compose

Automate docker compose files update

Docker Compose

In this scenario, we have to maintain a docker-compose file and we want to be sure that we always use the latest available docker.

Requirement

  1. Updatecli

  2. IDE

  3. A GitHub Personal Access Token

Updatecli Binary

Updatecli is a declarative dependency management tool. The application is available as a command line compatible with Linux/macOS/Windows. Updatecli is designed to work the same way, both from a local machine and a CI environment.

It can easily be installed using Homebrew by running the two following commands:

-> brew tap updatecli/updatecli
-> brew install updatecli

Additional installation instructions are available at www.updatecli.io/docs/prologue/installation.

IDE

The best way to write Updatecli manifest(s) is by using an IDE compatible with jsonschema hosted on schemastore.org, such as Vscode, Intellij, or Neovim. The full list of compatible IDE is available on www.schemastore.org.

For the IDE to load the correct Updatecli jsonschema, Updatecli manifest must have both a parent directory named "updatecli.d" and one of the file extension ".yaml", or ".yml". This provides auto-completion and validation out of the box.

For example, from VScode, typing [ctrl][space] should display a box with various suggestions

GitHub Personal Access Token

As we are going to interact with resources on Github such as github repository and GitHub packages, we need a personal access token. More information on the GitHub documentation creating-a-personal-access-token

Description

Docker image versions are identified using two methods, either the docker image tag or the docker image digest.

Tag

A docker image tag is useful when we want to have an easy to identify human-readable version like in nginx:1.17 where 1.17 is the tag. Unfortunately, a docker image tag can be easily overridden, whether on purpose or not. As when using the tag latest. This means that a docker image tag can’t be considered as a source of truth.

Digest

The second method to identify a docker image version is by using its digest like in nginx@sha256:8269a7352a7dad1f8b3dc83284f195bac72027dd50279422d363d49311ab7d9b where the string following nginx@sha256: is the digest. A digest is a hash referencing a specific version of a docker image tag, in this case, nginx:1.17. A digest is immutable, we can always rollback to a specific digest if needed. However, it’s not convenient to identify which version we’re referencing without a little further investigation.

The problem could be summarized as, "do we want to only use a well-defined docker image by using digests but with the cost of losing readability?" or, "are we accepting the risk of running an unwanted version by using image tags?"

It depends on how important we value readability versus uniqueness, but a useful rule of thumb is to consider the docker image build strategy. What’s the risk of running an unwanted version and how big is the chance of having to rollback. If we only build one image by version and that version is regularly updated then preferring readability by using docker image tag seems reasonable otherwise in any other scenarios it’s usually safer to use docker image digest.

Now that we know what to update and why, let see how to do it using updatecli.

Docker Image Tag

When we want to automate image tag updates, we have to identify the process used to define those tags, and if that process is automated. Then you can more than likely also automate the version update and this is where updatecli can play a role.

Using updatecli, you specify this process in the source stage, then if needed you check a condition and if the condition is satisfied then you update the version in your target file. Considering again, the example of regularly updating the Jenkins version. We know that a release is published every week on a maven repository, then a docker image build is published on DockerHub. Considering the build strategy, it would seem to be fine to use a docker image tag.

Let’s consider this docker compose file:

docker-compose.yaml

---
version: '3'
services:
  jenkins:
    image: jenkins/jenkins:2.245
    ports:
        - 8080:8080

Now we write an updatecli configuration with a source, conditions, and one target

updatecli.1.yaml

name: Example 1

sources:
  jenkins:
    name: "Get latest Jenkins version from maven repo"
    kind: maven
    transformers:
      # - addprefix: "v"
      - addsuffix: "-jdk11"
    spec:
      url: "repo.jenkins-ci.org"
      repository: "releases"
      groupid: "org.jenkins-ci.main"
      artifactid: "jenkins-war"

conditions:
  docker:
    name: "Docker Image Published on Registry"
    kind: dockerimage
    spec:
      image: "jenkins/jenkins"
      architecture: "amd64"

targets:
  imageTag:
    name: "Update docker-compose jenkins service"
    kind: yaml
    transformers:
      - addprefix: "jenkins/jenkins:"
    spec:
      file: "./docker-compose.yaml"
      key: "services.jenkins.image"

updatecli diff --config updatecli.1.yaml

Updatecli queries the maven repository 'releases' located on repo.jenkins-ci.org, to determine the latest version. If it finds one, then we add to the value retrieved, the prefix jenkins/jenkins: and the suffix -jdk11 as we are looking specifically for the java 11 version. We validate that the docker image exists on DockerHub and then we test that the value in the file 'docker-compose.yaml' for the key services.jenkins.image is correct. If not then we bump it.

If happy with the changes then we apply them by using apply instead of diff.

updatecli apply --config ./updatecli.1.yaml

Docker Image Digest

As explained earlier, sometimes it’s preferable to use the docker image digest, if we need to rely on a specific version, or if we want to easily rollback. Most of the configuration remains the same as the previous example. This time the source is dockerhub and we don’t test if an image exists on DockerHub as we’re already asking DockerHub for the latest image.

docker-compose.yaml

---
version: '3'
services:
  jenkins:
    image: jenkins/jenkins:2.245
    ports:
        - 8080:8080

updatecli.2.yaml

name: Example 2

sources:
  jenkins:
    kind: dockerdigest
    spec:
      image: "jenkins/jenkins"
      tag: "lts-jdk11"
      architecture: "amd64"

targets:
  imageTag:
    name: "jenkins/jenkins:lts-jdk11 docker digest"
    kind: yaml
    transformers:
      - addprefix: "jenkins/jenkins@sha256:"
    spec:
      file: "./docker-compose.yaml"
      key: "services.jenkins.image"

This time updatecli queries DockerHub to retrieve the digest for the docker image jenkins/jenkins:lts-jdk11. If it finds one, then we test that the value in the file 'docker-compose.yaml' for the key services.jenkins.image is correct, otherwise we change it.

Again, if the changes are acceptable then we apply them by using apply instead of diff.

  • updatecli diff --config updatecli.2.yaml

  • updatecli apply --config updatecli.2.yaml

Git/GitHub

Now that we have an easy way to update docker image version, we are missing a way to save, review, rollback those changes. For this, git is a excellent tool. Either, we directly commit and push to a git repository, or we use the GitHub workflow by pushing to a temporary branch. If using GitHub we can also submit our changes via a pull request.

docker-compose.yaml

---
version: '3'
services:
  jenkins:
    image: jenkins/jenkins:2.245
    ports:
        - 8080:8080

While the configuration remains similar to the earlier example, this time, a new element is introduced. We can define generic values in the file or read values from an environment variable like {{ requiredEnv "GITHUB_TOKEN" }}. The second major change is the 'scms' block which defines the github repository configuration.

updatecli.3.yaml

name: Example 3

sources:
  jenkins:
    name: "Get latest Jenkins version from maven repo"
    kind: "maven"
    transformers:
      # - addprefix: "v"
      - addsuffix: "-jdk11"
    spec:
      url: "repo.jenkins-ci.org"
      repository: "releases"
      groupid: "org.jenkins-ci.main"
      artifactid: "jenkins-war"

conditions:
  docker:
    name: "Docker Image Published on Registry"
    kind: "dockerimage"
    spec:
      image: "jenkins/jenkins"
      architecture: "amd64"

targets:
  imageTag:
    name: "Update docker-compose jenkins service"
    kind: "yaml"
    scmid: "updatecli/website"
    transformers:
      - addprefix: "jenkins/jenkins:"
    spec:
      file: "assets/code_example/docs/guides/docker-compose/docker-compose.yaml"
      key: "services.jenkins.image"

scms:
  updatecli/website:
    kind: "github"
    spec:
      user: "John"
      email: "info@updatecli.io"
      owner: "updatecli"
      repository: "website"
      token: '{{ requiredEnv "GITHUB_TOKEN" }}'
      username: "johnDoe"
      branch: "master"

And now you can use the same command than before

  • updatecli diff --config updatecli.3.yaml

  • updatecli apply --config updatecli.3.yaml

Conclusion

With this scenario, we saw how to automatically update a docker-compose file using custom strategies with updatecli. Updatecli is a small single binary that is suitable for use in your favorite CI environment.

Last modified on July 2, 2024
Edit this page on GitHub
← Autodiscovery
Automate with Jenkins →