Continuous Integration and Continuous Delivery (CICD) with Jenkins makes application delivery process easy and efficient without much manual effort. This blog post explains Spring Boot application deployment on Open Shift cluster (present on AWS) using Jenkins CICD pipeline. It has two major parts
- Setup Environment
- Run Pipeline
Prerequisites
For this, to work we need have Open Shift cluster running on AWS. If you are not sure how to do that, please look at this medium post for step by step instructions. Spring Boot application with Spring Data JPA and MySql. You can download my repository from Github.
Technologies
- OpenShift or OKD cluster on AWS
- Spring Boot 2.x
- Spring Data JPA 2.x
- MySql 5.7
- Jenkins 1.x
- SonarQube 6.7
- Nexus Artifactory 15.x
Setup Instructions
- Make sure OpenShift cluster is ready before proceeding further. All the steps and config are uploaded to my Bookstore Github repository
- SSH into openshift master node with AWS keypair you created while setting up the cluster
$ ssh -i <ssh_key.pem> centos@<openshift_master_hostname>3. Create three or more projects to deploy the Spring Boot application. Each one for dev, test, stage, and prod environments. You can skip some of these for simplicity. Project cicd will contain Jenkins, SOnarQube and Nexus pods
## Names can be anything, but make sure they are consistent across
oc new-project bookstore-dev --display-name="Bookstore - Dev"
oc new-project bookstore-stage --display-name="Bookstore - Stage"
oc new-project bookstore-stage --display-name="Bookstore - Stage"
oc new-project bookstore-stage --display-name="Bookstore - Stage"oc new-project cicd --display-name="cicd"4. Grant access to Jenkins service to create and deploy applications
# Grant Jenkins Access to Projects
oc policy add-role-to-user edit system:serviceaccount:cicd:jenkins -n bookstore-dev
oc policy add-role-to-user edit system:serviceaccount:cicd:jenkins -n bookstore-test
oc policy add-role-to-user edit system:serviceaccount:cicd:jenkins -n bookstore-stage
oc policy add-role-to-user edit system:serviceaccount:cicd:jenkins -n bookstore-prod5. If add-role-to-user step fails to execute the following the step and repeat the last step again
oc login -u system:admin
oc adm policy add-cluster-role-to-user cluster-admin <username>6. Make sure to my clone Github repository to get all the configuration files. You can also download just `templates` folder alone
7. Create jenkins-persistent pod in the cicd project by using jenkins-persistsnet-template.json template file
$ oc new-app -f templates/jenkins-persistsnet-template.json8. Once the pod is ready, make sure to create an external route to access it from the internet. See step 9 for details
9. To accept outside traffic to openshift cluster and jenkins, few steps need to be done
- Go to AWS Route 53 service to map external traffic to cluster and you should have a custom domain for this step. You can create on from Route 53 console.
- Create new
HostedZoneand provide your domain name as input. - Create a new record set with the following values
Name: jenkins
Alias: No
Value: <Master node IP Address>- Add inbound firewall rules to master node and worker security groups. See aws guide for more information on this
- Now in Jenkins pod, create an external route with the same domain name
- That’s it, you should be able to access Jenkins from outside
10. Login into Jenkins by clicking external URL and enter openshift credentials when required
11. Update your plugins and configure Maven, JDK and docker values inside Global Tool Configuration
12. Now the Jenkins is up and ready, set up sonarqubewith the following command
$ oc new-app -f templates/sonarqube-postgresql-template.yaml --param=SONARQUBE_VERSION=6.713. Create an external route as mentioned in step 9 with a different hostname
14. Login into sonarqube with default credentials and install Quality profiles if you don’t see any. Refer sonarqube docs for more information on this
15. Similarly setup Nexus repository with the following command
$ oc new-app -f templates/nexus3-template.yaml --param=NEXUS_VERSION=3.15.216. Create an external route as mentioned in step 9 with a different hostname
Run CICD pipeline using Jenkins
- I have Jenkisfile defined in my project with the required stages and steps added. Feel free to download and customize it
def version, mvnCmd = "mvn -s templates/cicd-settings-nexus3.xml"
pipeline
{
agent any
tools
{
maven 'M3'
}
stages
{
stage('Build App')
{
steps
{
git branch: 'openshift-aws', url: 'https://github.com/pavankjadda/BookStore.git'
script {
def pom = readMavenPom file: 'pom.xml'
version = pom.version
}
sh "mvn install -DskipTests=true"
}
}
stage('Test')
{
steps
{
echo "Test Stage"
sh "${mvnCmd} test -Dspring.profiles.active=test"
//step([$class: 'JUnitResultArchiver', testResults: '**/target/surefire-reports/TEST-*.xml'])
}
}
stage('Code Analysis')
{
steps
{
script
{
sh "${mvnCmd} sonar:sonar -Dsonar.host.url=http://sonarqube:9000 -DskipTests=true"
}
}
}
/*
stage('Archive App') {
steps {
sh "${mvnCmd} deploy -DskipTests=true -P nexus3"
}
}*/
stage('Create Image Builder') {
when {
expression {
openshift.withCluster() {
openshift.withProject(env.DEV_PROJECT) {
return !openshift.selector("bc", "bookstore").exists();
}
}
}
}
steps {
script {
openshift.withCluster() {
openshift.withProject(env.DEV_PROJECT) {
openshift.newBuild("--name=bookstore", "--image-stream=redhat-openjdk18-openshift:latest", "--binary=true")
}
}
}
}
}
stage('Build Image') {
steps {
sh "rm -rf ocp && mkdir -p ocp/deployments"
sh "pwd && ls -la target "
sh "cp target/bookstore-*.jar ocp/deployments"
script {
openshift.withCluster() {
openshift.withProject(env.DEV_PROJECT) {
openshift.selector("bc", "bookstore").startBuild("--from-dir=./ocp","--follow", "--wait=true")
}
}
}
}
}
stage('Create DEV') {
when {
expression {
openshift.withCluster() {
openshift.withProject(env.DEV_PROJECT) {
return !openshift.selector('dc', 'bookstore').exists()
}
}
}
}
steps {
script {
openshift.withCluster() {
openshift.withProject(env.DEV_PROJECT) {
def app = openshift.newApp("bookstore:latest")
app.narrow("svc").expose();
//http://localhost:8080/actuator/health
openshift.set("probe dc/bookstore --readiness --get-url=http://:8080/actuator/health --initial-delay-seconds=30 --failure-threshold=10 --period-seconds=10")
openshift.set("probe dc/bookstore --liveness --get-url=http://:8080/actuator/health --initial-delay-seconds=180 --failure-threshold=10 --period-seconds=10")
def dc = openshift.selector("dc", "bookstore")
while (dc.object().spec.replicas != dc.object().status.availableReplicas) {
sleep 10
}
openshift.set("triggers", "dc/bookstore", "--manual")
}
}
}
}
}
stage('Deploy DEV') {
steps {
script {
openshift.withCluster() {
openshift.withProject(env.DEV_PROJECT) {
openshift.selector("dc", "bookstore").rollout().latest();
}
}
}
}
}
stage('Promote to STAGE?') {
steps {
script {
openshift.withCluster() {
openshift.tag("${env.DEV_PROJECT}/bookstore:latest", "${env.STAGE_PROJECT}/bookstore:${version}")
}
}
}
}
stage('Deploy STAGE') {
steps {
script {
openshift.withCluster() {
openshift.withProject(env.STAGE_PROJECT) {
if (openshift.selector('dc', 'bookstore').exists()) {
openshift.selector('dc', 'bookstore').delete()
openshift.selector('svc', 'bookstore').delete()
openshift.selector('route', 'bookstore').delete()
}
openshift.newApp("bookstore:${version}").narrow("svc").expose()
openshift.set("probe dc/bookstore --readiness --get-url=http://:8080/actuator/health --initial-delay-seconds=30 --failure-threshold=10 --period-seconds=10")
openshift.set("probe dc/bookstore --liveness --get-url=http://:8080/actuator/health --initial-delay-seconds=180 --failure-threshold=10 --period-seconds=10")
}
}
}
}
}
}
}2. Go to Jenkins home page and click on the new item then enter <project name> and select Multibranch pipeline from the list
3. From home page, select BookStore job and Configure it with git repository for this pipeline
4. Scan Multibranch pipeline to detect branches (shown below) and click on the branch to execute it.
5. Once deployment went through, you should be able to see the following flow on Blue Ocean UI (Ships with Jenkins)
All the templates, config files can be found here https://github.com/pavankjadda/BookStore/tree/openshift-aws
