Building the CI/CD of the Future, Install and configure Jenkins
In this tutorial, I will share my experience as a DevOps engineer at Cloudify.co, this is the fifth post of the tutorial in which I will describe how to install Jenkins with helm, configure it, use a persistent volume, and do daily Jenkins backups.
Building the CI/CD of the Future published posts:
- Introduction
- Creating the VPC for EKS cluster
- Creating the EKS cluster
- Adding the Cluster Autoscaler
- Add Ingress Nginx and Cert-Manager
- Install and configure Jenkins
- Create your first pipeline
Let’s start.
What is Jenkins?
Jenkins is a self-contained, open source automation server which can be used to automate all sorts of tasks related to building, testing, and delivering or deploying software.
https://jenkins.io/doc/
Install Jenkins to EKS cluster
To install Jenkins to our EKS cluster, I will use this helm chart and my custom jenkins-values.yaml to override the defaults.
Review the configuration of jenkins-values.yaml
Ingress definition
ingress:
enabled: true
apiVersion: "extensions/v1beta1"
labels: {}
annotations:
kubernetes.io/ingress.class: nginx
cert-manager.io/cluster-issuer: "letsencrypt-prod"
hostName: jenkins.eks.cicd-future.com
tls:
- secretName: jenkins-tls-prod
hosts:
- jenkins.eks.cicd-future.com
We enabled the Ingress rule and after Jenkins is installed, it will be accessible as jenkins.eks.cicd-future.com, and Let’s Encrypt certificate will be obtained for this domain. You can read about the ingress and certification mechanism in detail in the previous post.
Backup
## Backup cronjob configuration
## Ref: https://github.com/maorfr/kube-tasks
backup:
# Backup must use RBAC
# So by enabling backup you are enabling RBAC specific for backup
enabled: true
# Used for label app.kubernetes.io/component
componentName: "backup"
# Schedule to run jobs. Must be in cron time format
# Ref: https://crontab.guru/
schedule: "0 3 * * *"
labels: {}
annotations: {}
# Example for authorization to AWS S3 using kube2iam
# Can also be done using environment variables
# iam.amazonaws.com/role: "jenkins"
image:
repository: "maorfr/kube-tasks"
tag: "0.2.0"
# Additional arguments for kube-tasks
# Ref: https://github.com/maorfr/kube-tasks#simple-backup
extraArgs: []
existingSecret:
jenkinss3:
awsaccesskey: jenkins_aws_access_key
awssecretkey: jenkins_aws_secret_key
env:
- name: "AWS_REGION"
value: "eu-east-1"
resources:
requests:
memory: 1Gi
cpu: 1
limits:
memory: 2Gi
cpu: 1
destination: "s3://jenkins-backups"
To use backups we using kube-tasks, it’s a simple mechanism that copy files of /var/jenkins_home (Jenkins master POD) folder on daily basis to s3 bucket, using the Kubernetes scheduled job.
To enable this mechanism you need to define your backup schedule in cronjob format like: ‘0 3 * * *’ and to have access to s3 bucket, using kube2iam or in my case, I created a specific user in AWS with access to S3. If you using AWS user like I do you need to store the credentials (aws_access_key / aws_secret_key) of this user as kubernetes secret, you can create a secret by running this command:
$ kubectl create secret generic jenkinss3 /
--from-literal=jenkins_aws_access_key=ACCESS_KEY /
--from-literal=jenkins_aws_secret_key=SECRET_KEY
ACCESS_KEY is aws_access_key you got from your user, after choosing programmatic access and exporting creds. SECRET_KEY is an aws_secret_key you exported.
The secret name is jenkinss3:
This mechanism worked for a couple of months till I found that 3 last jobs in a failed state and logs revealed ‘error in Stream: command terminated with exit code 1 src: file:”, you can read about this issue here. Failure occurs when files in /var/jenkins_home folder changing during upload to s3 bucket. This problem not solved, so my solution was to build this simple scheduled job which will archive /var/jenkins_home folder and uploads it to s3 on a daily basis.
You can read more about this solution here.
Persistent volume
persistence:
enabled: true
## A manually managed Persistent Volume and Claim
## Requires persistence.enabled: true
## If defined, PVC must be created manually before volume will be bound
existingClaim:
## jenkins data Persistent Volume Storage Class
## If defined, storageClassName: <storageClass>
## If set to "-", storageClassName: "", which disables dynamic provisioning
## If undefined (the default) or set to null, no storageClassName spec is
## set, choosing the default provisioner. (gp2 on AWS, standard on
## GKE, AWS & OpenStack)
##
storageClass:
annotations: {}
accessMode: "ReadWriteOnce"
size: "8Gi"
volumes:
# - name: nothing
# emptyDir: {}
mounts:
# - mountPath: /var/nothing
# name: nothing
# readOnly: true
We need a persistent volume for Jenkins to store all configuration and make sure the data not lost after POD restarted or worker node is rebooted.
It will use the default storage class:
if not selected otherwise, in my case it’s AWS EBS.
Set of plugins I am using
# List of plugins to be install during Jenkins master start
installPlugins:
- kubernetes:1.25.3
- workflow-job:2.38
- workflow-aggregator:2.6
- credentials-binding:1.23
- git:4.2.2
- configuration-as-code:1.39
- greenballs:1.15
- google-login:1.6
- github:1.29.5
- blueocean:1.23.1
- Jenkins Kubernetes plugin
- Plugins for the pipeline in Jenkins
- blueocean plugin which is very useful
- google-login plugin which lets you log in to Jenkins with your Google account
- simple-theme-plugin which makes Jenkins UI sexier, I personally use material-theme
Create and configure service-account
In order for Jenkins to be authenticated/authorized to the Kubernetes cluster, we need to create a service account. The service account is bound to cluster roles and can be granted certain cluster privileges.
In jenkins-values.yaml you must enable service-account creation:
serviceAccount:
create: true
name: jenkins
annotations: {}
Then you must grant to you service-account cluster-admin role by running this command:
$ kubectl create clusterrolebinding jenkinsrolebinding --clusterrole=cluster-admin --group=system:serviceaccounts:jenkins
Now Jenkins can access the Kubernetes cluster and create/update/delete different Kubernetes resources like pods/deployments/services …
You can read more about RBAC Authorization here.
Install Jenkins master on the cluster to jenkins namespace
Assuming your custom values.yaml for Jenkins helm chart is jenkins-values.yaml
$ kubectl create namespace jenkins
$ helm install jenkins -f jenkins-values.yaml stable/jenkins --namespace jenkins
Get the admin password of Jenkins
printf $(kubectl get secret --namespace jenkins jenkins -o jsonpath="{.data.jenkins-admin-password}" | base64 --decode);echo
The only thing left is to open Jenkins in the browser and login using the admin password you have from the previous step.
I the next post I will show you how to use a created service account on Jenkins and how to run a simple pipeline job that lists all pods in the Kubernetes cluster.
In this post, I explained how to install Jenkins, did an overview of my custom configuration, how to configure ingress rules, do daily backups, use persistent volume to store all the data, my set of plugins, how to create a service account with access to the Kubernetes cluster.
Please follow me on Twitter (@warolv)
I will save all configuration created in this tutorial in my Github
For consulting gigs you can reach me on Upwork
This post on my medium account: https://medium.com/@warolv/building-the-ci-cd-of-the-future-install-and-configure-jenkins-c17feea00ef7
Leave a comment