A few weeks ago I had to prepare our Jenkins for an internal cloud update. The update would take place over the weekend, so it had to be down on Saturday, 8 a.m. I didn’t want to wait until everyone was finished on Friday evening, so I had a look for an automatic shutdown procedure.
We use a Tomcat container where Jenkins is deployed as WAR. Therefore you cannot directly stop as there might be jobs running. Technically this shouldn’t be an issue but could lead to corrupted workspaces. So in order to have a “clean” shutdown, you have to enter Jenkins' quiet mode and after no more jobs are running you can stop Tomcat (via catalina.sh stop
).
Luckily Jenkins offers a remote API which lets you script most of the stuff, so the whole process was actually quite easy.
Authentication
If you are using a secured installation, API calls need to be authenticated. For sure I didn’t want to put my credentials in some script (as we’re using LDAP / ActiveDirectory login and the password is therefore somewhat sensitive), but Jenkins has an easy solution to that: API tokens. We have a service user anyway, so I quickly went to his user details page, generated an API token, and that was it. You can sent the username + api token in the HTTP basic authentication header with each request, to get all calls authenticated, even the ones which don’t support the API token within the query string (like the Jenkins quiet URL).
- Enter Jenkins' quiet mode.
Just hit the URLhttps://<jenkins-url>/quietDown
and you’re set. - Check if jobs are running
To check for running jobs we can use a combination of the XML API andsed
to extract the currently running jobs. Via<jenkins-url>/computer/api/xml?xpath=//busyExecutors
you get a response which looks like<busyExecutors>5</busyExecutors>
, which means currently are still 5 jobs running. Withsed -e 's,.*<busyExecutors>\([^<]*\)</busyExecutors>.*,\1,g'
we can extract the number within the XML and wait, if Jenkins returns anything else than 0. - No more jobs - then shutdown Tomcat
Business as usual, just callcatalina.sh stop
to finally shutdown your Tomcat.
Show me the code
Here is my little shell script, which I now use (obviously it could be prettier, but it does the job. You need to make sure, that curl has your SSL certificate and that you have catalina.sh in your path.
#!/bin/bash
CURL="/usr/bin/curl -s -X POST --user <user>:<apitoken>"
URL_QUIET="https://<jenkins-url>/quietDown"
URL_BUSY="https://<jenkins-url>/computer/api/xml?xpath=//busyExecutors"
$CURL $URL_QUIET
JOBS=`$CURL $URL_BUSY | /bin/sed -e 's,.*<busyExecutors>\([^<]*\)</busyExecutors>.*,\1,g'`
while [ $JOBS -gt 0 ]
do
echo "Still waiting for $JOBS jobs to finish. Sleeping 30 seconds"
sleep 30
JOBS=`$CURL $URL_BUSY | /bin/sed -e 's,.*<busyExecutors>\([^<]*\)</busyExecutors>.*,\1,g'`
done
catalina.sh stop
Have fun and drop me a note via Twitter if you find this post useful or have additions / corrections.
.sascha