Autodiscovery
Autodiscovery, auto pilot mode on
On this page
Description
In the lifecycle of a software project, dependencies are everywhere. From that third application used to lint our code, the tool used to build our documentation website, or the container image used to distribute our application. As the project grows, the number of dependencies grows as long. Some dependencies are context-dependent, and others are expressed using a standard data structure.
Updatecli is a great tool to handle both situations. It can either manage tailored update scenarios by using a manifest provided by a maintainer or automatically identify available update scenarios.
The latter approach removes the need to write manifest. This behavior is referred as "autodiscovery". When autodiscovery is enabled, Updatecli generates manifests in-memory before applying them (instead of reading manifests and applying).
A use case: a repository with a collection of Dockerfile`s.
To keep the Dockerfiles' `FROM
instructions up to date, writing one manifest file per Dockerfile
is not practical. With autodiscovery enabled, Updatecli scans the repository for all Dockerfile
and automatically tracks the FROM
instruction to propose updates if needed.
This would help us maintain our Dockerfile images without having to write any manifests.
Each autodiscovery scenario is handled by a crawler. The goal of the crawler is to parse recursively all files from a directory looking for the pattern and then try to generate as many updatecli manifests as it can. Once all manifests have been generated, we run them as we would do with custom manifest, by running updatecli diff or updatecli apply
The autodiscovery is enabled by default and should work out of the box just by running one of the following commands.
updatecli diff
updatecli show
updatecli apply
As in any opinionated way of working, a bit of adaptability is required. The next part of this document covers the different kinds of customization that can be used with autodiscovery.
Parameters
Autodiscovery could benefit from some customization, such as providing SCM configuration or defining pull request information. Some parameters are crawlers specific and others apply to all crawlers.
If a Updatecli manifest specifies the root key "autodiscovery" such as in the following example, then on top of the default autodiscovery, it enables an additional autodiscovery feature where we clone the epinio/helm-chart repository in a temporary location and then we look in that location for all Helm chart that specifies dependencies and we try to update them.
scms:
default:
kind: git
spec:
url: https://github.com/epinio/helm-charts.git
autodiscovery:
scmid: default
crawlers:
helm:
By default the autodiscovery looks for pattern from the local directory but we can also specify manifest from remote git repositories.
Scm
We assume that each crawler relies on the same scm configuration and if we can add more autodiscovery manifest to handle more repository
Pull Request
work in progress
Scenario to describe:
One pull request for everything
One pull request per crawler
One pull request per manifest from each crawlers
Crawlers
Crawlers implement common update scenarios.
Type
The purpose of a crawler is to generate manifest that updatecli can used. We identify two kinds of crawler, "standard", and "custom"
standard
A "standard" crawler is enabled by default but can be disabled on per case.
custom
A "custom" crawler is context specific, such as tailored to a company or a OSS project. By default it won’t be enabled by default.