Automatic updates with Renovate
Renovate Bot: Automatic updates, the missing piece to my GitOps workflow. This is how I manage my dependencies in my Github repository.
Renovate was the missing piece to my GitOps workflow: Managing updates! Its easy to setup and works seamlessly with FluxCD and Github. First I looked into the FluxCD to automate image updates to Git but found that scheduling to be lacklustre. I use Renovate as a dedicated tool for image updates, and let FluxCD handle the kubernetes deployments. It results in a clean GitOps workflow were I am able to review the updates before approving them, making it easier to plan by implementing small (minor version updates) changes continuously. Compared major changes by updating once a month.
Contents
Renovated GitOps Workflow
How do I use Renovate to apply K3s updates with FluxCD and Github?
- Renovate constantly scans my repo for out-of-date dependencies
- If it finds anything, Renovate opens a Pull Request to upgrade the version
- Review the Pull Request and merge it to upgrade the version
- Flux scans the repo, sees the update, and redeploys my applications
Renovate comes with a Dependency Dashboard where you can find all opened and pending updates, along with any Repository problems. It allows me to focus on the pull requests that Renovate generates. I will check changelogs and add them to the merge request for auditing. Flux will take care of the rest with its reconciliation loop.
Updating Nextcloud
Here we have a pull request on our dependency dashboard for Nextcloud:
Opening the Nextcloud WebUI shows us a notification update as well:
Version 31.0.7 features a host of fixes, and some changes: https://nextcloud.com/changelog/#31-0-7
After reading through the changelog and merging the pull request we see that nextcloud:31.0.7-apache was successfully deployed:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 55s default-scheduler Successfully assigned nextcloud/nextcloud-85bbd878d6-sfgk4 to node4-slytherin
Normal Pulling 55s kubelet Pulling image "nextcloud:31.0.7-apache"
Normal Pulled 45s kubelet Successfully pulled image "nextcloud:31.0.7-apache" in 10.538s (10.538s including waiting). Image size: 467806863 bytes.
Normal Created 45s kubelet Created container: nextcloud
Normal Started 45s kubelet Started container nextcloud
When we take a closer look we can see the Deployment scaling up the new ReplicaSet:
2m15s Normal Killing pod/nextcloud-d67b959cc-dkbwc Stopping container nextcloud
2m15s Normal SuccessfulDelete replicaset/nextcloud-d67b959cc Deleted pod: nextcloud-d67b959cc-dkbwc
2m27s Normal ScalingReplicaSet deployment/nextcloud Scaled up replica set nextcloud-85bbd878d6 to 1
2m15s Normal ScalingReplicaSet deployment/nextcloud Scaled down replica set nextcloud-d67b959cc to 0 from 1
When we login using the WebUI we can verify that the latest version was installed:
Checking the security of our private Nextcloud server we see that we are running the latest patch level with an A+ rating:
Updating Ghost CMS
Next up I wanted to update this Ghost CMS website with the following pull request that Renovate generated: Update ghost Docker tag to v5.129.1
After a quick look at the release notes I see that the update contains a new feature to use Linkedin profiles.
After merging the pull request I can verify the redeployment of Ghost in my k3s cluster by describing the ghost pod.
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 59m default-scheduler Successfully assigned rudolf-ghost/ghost-6587dcd674-vbg6l to node3-hufflepuff
Normal Pulling 59m kubelet Pulling image "ghost:5.129.1"
Normal Pulled 59m kubelet Successfully pulled image "ghost:5.129.1" in 16.497s (16.497s including waiting). Image size: 191877806 bytes.
Normal Created 59m kubelet Created container: ghost
Normal Started 59m kubelet Started container ghost
My Grafana K3S monitoring dashboard also logs the CPU spike that shows the redeployment on node3-hufflepuff.
Now when I check the author on https://roomofrequirement.nl/ I see my LinkedIn profile confirming the update!
Updating FluxCD
Renovate also supports updating Flux dependencies. By default, Renovate will check any files matching the following regular expression:
/(?:^|/)gotk-components\.ya?ml$/.
Here is an example of a pull request to update one of the FluxCD dependencies:
The FluxCD changelog mentions a patch release that updates the kustomize-controller to v1.6.1
After merging this pull request we can verify by printing the flux-system deployments:
NAME IMAGE
helm-controller ghcr.io/fluxcd/helm-controller:v1.3.0
image-automation-controller ghcr.io/fluxcd/image-automation-controller:v0.41.2
image-reflector-controller ghcr.io/fluxcd/image-reflector-controller:v0.35.2
kustomize-controller ghcr.io/fluxcd/kustomize-controller:v1.6.1 <- Update
notification-controller ghcr.io/fluxcd/notification-controller:v1.6.0
source-controller ghcr.io/fluxcd/source-controller:v1.6.2
We can go into more detail by describing the pod and see the actual event:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 11m default-scheduler Successfully assigned flux-system/kustomize-controller-5cd6c746c7-j8vjc to node1-gryffindor
Normal Pulling 11m kubelet Pulling image "ghcr.io/fluxcd/kustomize-controller:v1.6.1"
Normal Pulled 11m kubelet Successfully pulled image "ghcr.io/fluxcd/kustomize-controller:v1.6.1" in 3.297s (3.297s including waiting). Image size: 67373776 bytes.
Normal Created 11m kubelet Created container manager
Normal Started 11m kubelet Started container manager
Updating Plex
So far Renovate has provided pull requests for a number of things but handling updates for my Plex deployment comes with an issue. Namely how Renovate treats Docker versioning. After configuring the Renovate "versioning": "loose" we get the following pull request.
This update contains the following changes:
New PUBLIC PMS Version Available - 1.41.9.9961-46083195d
News
ITEMS ADDED:
(Web) Updated to 4.147.1
ITEMS FIXED:
(Music) Music files with embedded Release Type tags are failing to extract local metadata (PM-3751)
(Networking) Replaced enable ipv6 preference with client network. (PM-2912)
(Photos) GPS coordinate data was not getting saved to the database (PM-3584)
(Photos) GPS coordinates are not being serialized correctly (PM-3482)
(Statistics) Some bandwidth data was not being recorded (PM-3483)
(Transcode) Server could occasionally crash on request reporting progress (PM-3610)
(Transcode) Some forms of rotation metadata were not detected. (PM-3468)
When we monitor events after merging this pull request we see that Plex is deployed with using the latest version:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 25s default-scheduler Successfully assigned media/plex-7b69d4887d-db72r to node4-slytherin
Normal Pulling 25s kubelet Pulling image "plexinc/pms-docker:1.41.9.9961-46083195d"
Normal Pulled 20s kubelet Successfully pulled image "plexinc/pms-docker:1.41.9.9961-46083195d" in 5.648s (5.648s including waiting). Image size: 161588166 bytes.
Normal Created 20s kubelet Created container: plex
Normal Started 19s kubelet Started container plex
Login in on our Plex instance we can verify the latest version:
How I configured Renovate
I chose to install the Mend Renovate App hosted on GitHub.com known as a repository configuration. This configuration is hosted at the root level of my repository. This setup comes with a renovate.json file that contains information on how the repository should be treated.
- First, navigate to https://github.com/apps/renovate and select the Install button:
- Configure Repository access:
- Activate Renovate by merging the Pull Request:
- Configure kubernetes datasource and directory in the rootfolder:
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": ["config:recommended"],
"kubernetes": {
"managerFilePatterns": ["/apps/base/.+\\.yaml$/"]
},
"packageRules": [
{
"matchPackageNames": ["plexinc/pms-docker"],
"matchDatasources": ["docker"],
"versioning": "loose"
},
{
"matchPackageNames": ["vaultwarden/server"],
"matchDatasources": ["docker"],
"versioning": "loose"
},
{
"matchDatasources": ["docker"],
"matchPackageNames": [
"mariadb",
"mysql",
"redis",
"mongo"
],
"enabled": false
}
]
}
Kubernetes is not configured by default. So we need to edit our renovate-schema.json to setup the .yaml naming convention for kubernetes YAML files and add the directory in my Monorepo. I configured that by setting the "managerFilePatterns" to match all the YAML files in the "/apps/base/" directory.
Plex and Vaultwarden don't want to play nice with the default strict versioning, so to combat that I configured a custom versioning "loose" to give Renovate more flexibility to identify the correct version.
This is actually one of the common practices mentioned in the Mend.io Renovate configuration.
The last task I configured was to stop Renovate from updating databases. The reason for that is that not all deployments support every version of the database. Case in point is Nextcloud:
Database: MySQL 8.0 / 8.4 or MariaDB 10.6/ 10.11 (recommended) / 11.4
Wrapping up
Renovate has been a great addition to my GitOps workflow. Setup is very simple, the Renovate docs are great. The ability to update both my deployments like Nextcloud, Ghost CMS, Plex and Vaultwarden along with FluxCD dependencies is amazing. Receiving an email from the renovatebot with the package update is the icing on the cake. It motivates me to follow up the change logs, actually tracking the changes. Finding out about new features that I would otherwise not even notice. I highly recommend Renovate!
Sources
Common Practices for Renovate Configuration
Does anyone upgrade Flux using Renovate?