Creating a local CI/CD pipeline from nothing
This is going to be a long and continuously updated story and a long overdue one.
I often ask myself why I haven’t yet experimented with the tech I use. There is no answer other than procrastination. So lets dive right in.
- Learning a bit of python scripting, we most likely will need it at some point
- Setup a local k8s cluster
- Where are you going to be stuffing your images? Harbor
- Get Jenkins up and running in k8s
- Get Sonar in there
- A simple pipeline deploying to k8s (we really need another cluster)
- Terraform to get our servers up and running the same, everytime. This somehow solves the calico problem too.
- Using Rancher to deal with calico
- Some ansible use somewhere perhaps
So, somethings have gone naturally and I can’t yet overcome some obstacles.
- Calico is a nightmare to setup. There is something wrong with its autodetection and even if correct adaptor is given, we get no internet access for some reason. Pods cant reach core-dns service ip. I’ve setup a cluster for calico about 15 times I think. Flannel works immediately so there’s something else calico needs. After Rancher, I will check again.
- Jenkins slaves and I do not get along. Slaves are automatically created on demand, everything is great but unless I reinstall gradle and maven to the slaves at launch of pipeline we get nothing. I created (and used, I double checked) modified slave image with both gradle and maven but it says no to both. I downloaded someone elses images from docker hub and I can see maven in the images but when launched from Jenkins, it’s a no. I’m not going to go through the hassle of creating a static node just for my test builds so master builds it is for now.
- We need an external or NFS client like storage solution. Hostpath PVs are going to be a problem.
- So If we have Jenkins running on a k8s pod we cant use Docker build even if we install which sucks. I’m don’t see much point of using Jenkins on k8s.