Creating a local CI/CD pipeline from nothing

This is going to be a long and continuously updated story and a long overdue one.

I often ask myself why I haven’t yet experimented with the tech I use. There is no answer other than procrastination. So lets dive right in.

• Learning a bit of python scripting, we most likely will need it at some point
• Setup a local k8s cluster
• Where are you going to be stuffing your images? Harbor
• Get Jenkins up and running in k8s
• Get Sonar in there
• A simple pipeline deploying to k8s (we really need another cluster)
• Terraform to get our servers up and running the same, everytime. This somehow solves the calico problem too.
• Using Rancher to deal with calico
• Some ansible use somewhere perhaps

Problems

So, somethings have gone naturally and I can’t yet overcome some obstacles.

• Calico is a nightmare to setup. There is something wrong with its autodetection and even if correct adaptor is given, we get no internet access for some reason. Pods cant reach core-dns service ip. I’ve setup a cluster for calico about 15 times I think. Flannel works immediately so there’s something else calico needs. After Rancher, I will check again.
• Jenkins slaves and I do not get along. Slaves are automatically created on demand, everything is great but unless I reinstall gradle and maven to the slaves at launch of pipeline we get nothing. I created (and used, I double checked) modified slave image with both gradle and maven but it says no to both. I downloaded someone elses images from docker hub and I can see maven in the images but when launched from Jenkins, it’s a no. I’m not going to go through the hassle of creating a static node just for my test builds so master builds it is for now.
• We need an external or NFS client like storage solution. Hostpath PVs are going to be a problem.
• So If we have Jenkins running on a k8s pod we cant use Docker build even if we install which sucks. I’m don’t see much point of using Jenkins on k8s.