Docker Image: new Exit and Restart Behavior for Controllers
The Jenkins project provides Docker images for controllers (and more).
Beginning with Jenkins 2.344 released April 18, 2022 and Jenkins 2.332.3 released May 04, 2022, the behavior of the "Exit" and "Restart" lifecycle of the controller image jenkins/jenkins changed.
|
TL;DR; Ensure that you have a Container Restart Policy For quick readers: check the How to Add a Container Restart Policy section for immediate actions to be taken. |
Application Lifecycle in a Container World
Jenkins previously restarted itself after upgrading plugins or via the /restart or /safeRestart endpoints by calling exec(2).
This was fragile and exposed users to a variety of bugs.
The ExitLifecycle implementation allows the main process to exit normally before starting it again from scratch,
relying on the exit code to signal to the container orchestrator that the container ought to be restarted rather than to remain shut down.
Avoiding exec(2) from Java code, with all the associated complexity of closing file handles and making low-level system calls via JNA, eliminates exposure to a category of bugs.
You can read more about this on the following issues: jenkinsci/jenkins#5899 and jenkinsci/docker#1268.
Delegate Restarts to the Container Engine
Since the Jenkins Weekly 2.344 and the Jenkins LTS 2.332.3, the Docker image for Jenkins controller uses the new lifecycle ExitLifecycle by default.
It means that when the Jenkins controller triggers any "restart" or "exit" event, then its container exits.
If you are running Jenkins in a controller as a container, you should add a "container restart policy" different than none to ensure that the container is restarted automatically.
If you are running Jenkins in production in a container, chances are great that you already have a restart policy set and the change will be transparent.
But we updated the jenkinsci/docker documentation to mention that the flag --restart=on-failure is set when starting a Jenkins controller container.
How to Add a Container Restart Policy
If you do not have a container restart policy defined, please find the different following methods depending on your container orchestrator:
-
Docker Engine, Docker Swarm: add the flag
--restart=on-failureto thedocker run <…> jenkins/jenkinscommand.No need to create a new container: you can use the command docker updateto update the restart policy of an existing container. -
Docker Compose: update your
docker-compose.ymlfile to specify therestart: on-failurekeyword (or thedeploy.restart_policyif usingdocker compose deployto a Docker Swarm cluster) -
Kubernetes:
-
If you are using the official Jenkins helm-chart, make sure that you are using at least the version
3.10.3of the Helm chart and you’re good to go, as per jenkinsci/helm-charts#529 -
Otherwise, ensure that you configure the Deployment for Jenkins to specify the
OnFailurerestart policy for the pods. Please note that it’s the default policy if unspecified.
-
-
Hashicorp’s Nomad: Ensure that you specify a non-empty
restartstanza. Please note that the default stanza behavior behaves as expected: Jenkins is restarted.
|
Damien DUPORTAL
Damien is the Jenkins Infrastructure officer and a software engineer at CloudBees working as a Site Reliability Engineer for the Jenkins Infrastructure project. Not only he is a decade-old Hudson/Jenkins user but also an open-source citizen who participates in Updatecli, Asciidoctor, Traefik and many others. |
|
Basil Crow
Basil is a long-time Jenkins user and contributor, a Jenkins core maintainer, and the maintainer of the Email Extension, Timestamper, and Swarm plugins (among others). Basil enjoys working on open source software in his free time. |