Guide: docker compose

Automate docker compose files update

Docker Compose

In this scenario, we have to maintain a docker-compose file and we want to be sure that we always use the newest docker images available.

Requirement

  1. Updatecli

  2. IDE

  3. A GitHub Personal Access Token

Updatecli Binary

Updatecli is a declarative dependency management tool. The application is available as a command line compatible with Linux/macOS/Windows. Updatecli is designed to work the same way, both from a local machine and a CI environment.

It can easily be installed using Homebrew by running the two following commands:

-> brew tap updatecli/updatecli
-> brew install updatecli

Additional installation instructions are available at www.updatecli.io/docs/prologue/installation.

IDE

The best way to write Updatecli manifest(s) is by using an IDE compatible with jsonschema hosted on schemastore.org, such as Vscode, Intellij, or Neovim. The full list of compatible IDE is available on www.schemastore.org.

For the IDE to load the correct Updatecli jsonschema, Updatecli manifest must have both a parent directory named "updatecli.d" and one of the file extension ".yaml", or ".yml". This provides auto-completion and validation out of the box.

For example, from VScode, typing [ctrl][space] should display a box with various suggestions

GitHub Personal Access Token

As we are going to interact with resources on Github such as github repository and GitHub packages, we need a personal access token. More information on the GitHub documentation creating-a-personal-access-token

Description

Docker image versions can be identified using two methods, either we use the docker image tag or the docker image digest.

Tag

A docker image tag is useful when we want to have an easy to identify human-readable version like in nginx:1.17 where 1.17 is the tag, unfortunately, a docker image tag can be easily overridden, on purpose or not. Like when using the tag latest. This means that a docker image tag can’t be considered as a source of truth.

Digest

The second method to identify a docker image version is by using its digest like in nginx@sha256:8269a7352a7dad1f8b3dc83284f195bac72027dd50279422d363d49311ab7d9b where 8269a7352a7dad1f8b3dc83284f195bac72027dd50279422d363d49311ab7d9b is the digest. A digest is a hash referencing a specific version of a docker image tag like in this case nginx:1.17. A digest is immutable, we can always rollblack to a specific digest if need. But we must admit that it’s not convenient to identify which version we are referencing to without having to dig into it.

The problem here could be resumed by "Do we want to only use well-defined docker image by using digests but with the cost of losing readability?" or "Are we ok to take the risk of running an unwanted version by using image tags?".

It depends on how important we value readability versus uniqueness, but a rule of thumb is to consider the docker image build strategy, what’s the risk of running an unwanted version and how big is the chance of having to rollback. If we only build one image by version and that version is regularly updated then preferring readability by using docker image tag seems ok-ish otherwise in any other scenarios It’s usually safer to use docker image digest.

Now that we know what we need to update and why, let see how we can do it using updatecli.

Docker Image Tag

When we want to automate image tag updates, we have to identify the process used to define those tags, and if that process is automated, then you can more than likely also automate version update that were updatecli can play a role.

In updatecli, you specify this process in the source stage, then if needed you apply some condition and finally if everything goes well then you update the version in your target file. So let take this example where we want to regularly update the Jenkins version, we know that a weekly release is published every week on a maven repository, then a docker image build and published on DockerHub. Considering the build strategy, it seems fine to use docker image tag.

Let’s consider this docker compose file:

docker-compose.yaml

---
version: '3'
services:
  jenkins:
    image: jenkins/jenkins:2.245
    ports:
        - 8080:8080

Now we write an updatecli configuration that contains a source, a conditions, and one target

updatecli.1.yaml

name: Example 1

sources:
  jenkins:
    name: "Get latest Jenkins version from maven repo"
    kind: maven
    transformers:
      # - addprefix: "v"
      - addsuffix: "-jdk11"
    spec:
      url: "repo.jenkins-ci.org"
      repository: "releases"
      groupid: "org.jenkins-ci.main"
      artifactid: "jenkins-war"

conditions:
  docker:
    name: "Docker Image Published on Registry"
    kind: dockerimage
    spec:
      image: "jenkins/jenkins"
      architecture: "amd64"

targets:
  imageTag:
    name: "Update docker-compose jenkins service"
    kind: yaml
    transformers:
      - addprefix: "jenkins/jenkins:"
    spec:
      file: "./docker-compose.yaml"
      key: "services.jenkins.image"

updatecli diff --config updatecli.1.yaml

Updatecli will query the maven repository 'releases' located on repo.jenkins-ci.org, to have the latest version. If it finds one, then we add to the value retrieved, the prefix jenkins/jenkins: and the suffix -jdk11 as we are looking specifically for the java 11 version. We validate that docker image effectively exist on DockerHub and then we test if the value in the file 'docker-compose.yaml' for the key services.jenkins.image is correct otherwise we bump it.

If we are good with the changes then we apply them by using apply instead of diff.

updatecli apply --config ./updatecli.1.yaml

Docker Image Digest

As explained earlier, sometimes we prefer to use docker image digest, if we need to rely on a specific version or if we want to easily rollback. While most of the configuration remains the same as in the previous example, this time the source will be dockerhub and we won’t test if an image already exists on DockerHub as we already ask DockerHub for the latest image available.

docker-compose.yaml

---
version: '3'
services:
  jenkins:
    image: jenkins/jenkins:2.245
    ports:
        - 8080:8080

updatecli.2.yaml

name: Example 2

sources:
  jenkins:
    kind: dockerdigest
    spec:
      image: "jenkins/jenkins"
      tag: "lts-jdk11"
      architecture: "amd64"

targets:
  imageTag:
    name: "jenkins/jenkins:lts-jdk11 docker digest"
    kind: yaml
    transformers:
      - addprefix: "jenkins/jenkins@sha256:"
    spec:
      file: "./docker-compose.yaml"
      key: "services.jenkins.image"

This time updatecli queries DockerHub to retrieve the digest for the docker image jenkins/jenkins:lts-jdk11. If it finds one, then we test if the value in the file 'docker-compose.yaml' for the key services.jenkins.image is correct otherwise we change it.

Again if we are ok with the changes then we apply them by using apply instead of diff.

  • updatecli diff --config updatecli.2.yaml

  • updatecli apply --config updatecli.2.yaml

Git/GitHub

Now that we have an easy way to update docker image version, we are missing a way to save, review, rollback those changes, and git for this is a tremendous tool. Either we directly commit and push to a git repository or we use the GitHub workflow by pushing to a temporary branch. If we are using GitHub we can also we submit our changes via a pull request which can then be approved.

docker-compose.yaml

---
version: '3'
services:
  jenkins:
    image: jenkins/jenkins:2.245
    ports:
        - 8080:8080

While the configuration remains quite similar to our previous example, this time we introduce a new element. We can define generic values in file or read values from an environment variable like {{ requiredEnv "GITHUB_TOKEN" }}. The second major change is the 'scms' block which should be quiet obvious and defines the github repository configuration.

updatecli.3.yaml

name: Example 3

sources:
  jenkins:
    name: "Get latest Jenkins version from maven repo"
    kind: "maven"
    transformers:
      # - addprefix: "v"
      - addsuffix: "-jdk11"
    spec:
      url: "repo.jenkins-ci.org"
      repository: "releases"
      groupid: "org.jenkins-ci.main"
      artifactid: "jenkins-war"

conditions:
  docker:
    name: "Docker Image Published on Registry"
    kind: "dockerimage"
    spec:
      image: "jenkins/jenkins"
      architecture: "amd64"

targets:
  imageTag:
    name: "Update docker-compose jenkins service"
    kind: "yaml"
    scmid: "updatecli/website"
    transformers:
      - addprefix: "jenkins/jenkins:"
    spec:
      file: "assets/code_example/docs/guides/docker-compose/docker-compose.yaml"
      key: "services.jenkins.image"

scms:
  updatecli/website:
    kind: "github"
    spec:
      user: "John"
      email: "info@updatecli.io"
      owner: "updatecli"
      repository: "website"
      token: '{{ requiredEnv "GITHUB_TOKEN" }}'
      username: "johnDoe"
      branch: "master"

And now you can use the same command than before

  • updatecli diff --config updatecli.3.yaml

  • updatecli apply --config updatecli.3.yaml

Conclusion

During this scenario, we saw how to automatically update docker-compose file using custom strategies with Updatecli. Updatecli is a small tool that can be used from your favorite CI environment.

Now we can replace our docker-compose file by any other YAML file to automate YAML update.

Last modified on February 27, 2023
Edit this page on GitHub
← Autodiscovery
Automate with Jenkins →