Run Jenkins in k8s

So yeah, why not use Jenkins in a k8s? Lets get one up an running.

First we need to create an image with some plugins we want pre installed. Normally if we were to install to a VM with internet connection, they could be installed via the Jenkins UI.

Our Dockerfile is like follows. I went a bit overboard with the plugins but I do want to try them all out. Plugins are here

Create a file called Dockerfile with the below content. We will be using the default jenkins image as a base.

Build our image and stuff it into our Harbor image repo.

docker build -t 192.168.56.109:443/core-sys/jenkins-mod:v1 .

So the image is gigantic, but we will look into that later :)

Lets get Jenkins running with our new image. Below is the yaml we will use to deploy Jenkins. I’ll explain each yml seperated by — -

  • We first create a cluster role binding for the default service account with cluster-admin permission for jenkins to use. Sounds a bit over-kill, yes, but we’ll fix that later.
  • Then a persistent vol in master node within the host path to pass as vol to Jenkins
  • Then the PVC to claim above vol
  • A nodeport service that allows us to use port 30007 to point to Jenkins 8080 port.
  • The Jenkins Deploy yaml itself, describing the port we will use, Java env options, our modified image

After running above with kubectl apply -f . running below should get you something like the bottom image.

kubectl get pod,svc,pvc,pv,sa,clusterrolebinding

So we can open our jenkins now from our master ip: http://192.168.56.105:30007/configureSecurity/

The first thing would be to change the security to user based like below.

seup security first
our first user

Once thats out of the way. Lets set the workers to … work.

Select Kubernetes option from cloud
click to button to add details
# K8s url
kubectl config view --minify | grep server | cut -f 2- -d ":" | tr -d " "
# Server certificate key
kubectl get secret $(kubectl get sa default -o jsonpath={.secrets[0].name}) -o jsonpath={.data.'ca\.crt'} | base64 --decode
# jenkins url is the servicename with the cluster ip port we used.

Credential provider part should be selected as k8s SA and global. We then click add.

clicking test connections yields success.

We then have to add pod templates for jenkins slaves.

So lets give it a test with a new job: http://192.168.56.105:30007/newJob

Added just a single sleep command and saved.

Before testing, lets tick a box in the build config to be able to run the same unchanged build multiple times. Normally Jenkins doesn’t allow such.

Started a few builds together from below

yay

Thats about it. Thanks for reading.

if you want you can modify the slave images too.

use in configurations like: 192.168.56.109:443/core-sys/jenkins/jnlp-slavemod:v1.0Dockerfile:
FROM jenkinsci/jnlp-slave
USER root
RUN apt-get update \
&& apt-get install -y maven \
&& apt-get install -y gradle

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Yiğit İrez

Yiğit İrez

Let’s talk devops, automation and architectures, everyday, all day long. https://www.linkedin.com/in/yigitirez/