Jenkins | Build and Deploy a Groovy App
Introduction
Using Jenkins as an automation server for your development, you can automate such repeating tasks as testing and deploying your app.
Starting with a sample Groovy App (a simple calculator) with tests, you will learn how to integrate your app in Jenkins and build a pipeline, so that Jenkins runs the desired tasks every time, you change the code.
Prepare the sources
Clone the sample repository from Github.
You should clone the demo repository into you demo account, because you may change some file during this post., and you will not get write permissions for the demo repository.
Also, clone the repository to your local machine to see what our demo app looks like.
$ git clone https://github.com/jenkins-toolbox/SampleApp_GroovyCalculator Cloning into 'SampleApp_GroovyCalculator'... remote: Enumerating objects: 194, done. remote: Counting objects: 10 remote: Compressing objects: 10 remote: Total 194 (delta 44), reused 137 (delta 23), pack-reused 0 Receiving objects: 10 Resolving deltas: 10Go into the new create folder
$ cd SampleApp_GroovyCalculator/ $ ls Jenkinsfile README.md bin build.gradle gradlew src Makefile SampleCalculator build gradle settings.gradleThe first task, Jenkins will do in our pipeline: build your app
$ ./gradlew buildBecause it’s the first time you start
gradlew
, the required software will be downloaded:First: the current Gradle Version (Gradle is the Build Tool used by Groovy Projects)
Downloading https://services.gradle.org/distributions/gradle-6.2.1-bin.zip ………1 Welcome to Gradle 6.2.1! Here are the highlights of this release: - Dependency checksum and signature verification - Shareable read-only dependency cache - Documentation links in deprecation messages For more details see https://docs.gradle.org/6.2.1/release-notes.html Starting a Gradle Daemon, 2 stopped Daemons could not be reused, use --status for detailsAfter this, your app will be tested
> Task :test Calculator02Spec > two plus two should equal four PASSED Calculator01Spec > add: 2 + 3 PASSED Calculator01Spec > subtract: 4 - 3 PASSED Calculator01Spec > multiply: 2 * 3 PASSED BUILD SUCCESSFUL in 34s 5 actionable tasks: 5 executedPerform the build again
No download is required. The build is much quicker.
$ ./gradlew build BUILD SUCCESSFUL in 1s 5 actionable tasks: 5 up-to-dateNow, test our app:
./gradlew clean test > Task :test Calculator02Spec > two plus two should equal four PASSED Calculator01Spec > add: 2 + 3 PASSED Calculator01Spec > subtract: 4 - 3 PASSED Calculator01Spec > multiply: 2 * 3 PASSED BUILD SUCCESSFUL in 4s 5 actionable tasks: 5 executedCreate a Jenkins Pipeline
Start by clicking on the BlueOcean menu item.
demo repositoryNext, we select the demo repository SampleApp_GroovyCalculator
Click on Create Pipeline and after a few seconds, the pipeline is created.
Immediately after creating the pipeline, Jenkins is starting the pipeline and all steps included.
Jenkins | Cookbook
Working with VS Code
Validate Jenkins File
Install VS Code Plugin Jenkins Pipeline Linter Connector
Add configuration in .vscode/settings.json
"jenkins.pipeline.linter.connector.crumbUrl": "<JENKINS_URL>/crumbIssuer/api/xml?xpath=concat(//crumbRequestField "jenkins.pipeline.linter.connector.user": "<USERNAME>", "jenkins.pipeline.linter.connector.pass": "<PASSWORD>", "jenkins.pipeline.linter.connector.url": "<JENKINS_URL>/pipeline-model-converter/validate",
Replace <USERNAME>
, <PASSWORD>
and <JENKINS_URL>
with your values, for example
"jenkins.pipeline.linter.connector.crumbUrl": "http://localhost:8080/crumbIssuer/api/xml?xpath=concat(//crumbRequestField "jenkins.pipeline.linter.connector.user": "admin", "jenkins.pipeline.linter.connector.pass": "secret", "jenkins.pipeline.linter.connector.url": "http://localhost:8080/pipeline-model-converter/validate",
Working with Jenkins Client (CLI)
Download Client
wget localhost:8080/jnlpJars/jenkins-cli.jar
Working with Plugins
Create aPlugin
mkdir SamplePlugin cd SamplePlugin
mvn -U archetype:generate -Dfilter="io.jenkins.archetypes:"
mvn -U archetype:generate -Dfilter="io.jenkins.archetypes:global-configuration-plugin" [INFO] Scanning for projects... Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-metadata.xml Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-metadata.xml Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-metadata.xml (14 kB at 32 kB/s) Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-metadata.xml (20 kB at 44 kB/s) Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-archetype-plugin/maven-metadata.xml Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-archetype-plugin/maven-metadata.xml (918 B at 18 kB/s) [INFO] [INFO] ------------------< org.apache.maven:standalone-pom >------------------- [INFO] Building Maven Stub Project (No POM) 1 [INFO] --------------------------------[ pom ]--------------------------------- [INFO] [INFO] >>> maven-archetype-plugin:3.1.2:generate (default-cli) > generate-sources @ standalone-pom >>> [INFO] [INFO] <<< maven-archetype-plugin:3.1.2:generate (default-cli) < generate-sources @ standalone-pom <<< [INFO] [INFO] [INFO] --- maven-archetype-plugin:3.1.2:generate (default-cli) @ standalone-pom --- [INFO] Generating project in Interactive mode [INFO] No archetype defined. Using maven-archetype-quickstart (org.apache.maven.archetypes:maven-archetype-quickstart:1.0)
Choose archetype: 1: remote -> io.jenkins.archetypes:global-configuration-plugin (Skeleton of a Jenkins plugin with a POM and an example piece of global configuration.) Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains): : 1 Choose io.jenkins.archetypes:global-configuration-plugin version: 1: 1.2 2: 1.3 3: 1.4 4: 1.5 5: 1.6 Choose a number: 5: [INFO] Using property: groupId = unused Define value for property 'artifactId': com.examples.jenkins.plugins Define value for property 'version' 1.0-SNAPSHOT: : [INFO] Using property: package = io.jenkins.plugins.sample Confirm properties configuration: groupId: unused artifactId: com.examples.jenkins.plugins version: 1.0-SNAPSHOT package: io.jenkins.plugins.sample Y: : y
[INFO] ---------------------------------------------------------------------------- [INFO] Using following parameters for creating project from Archetype: global-configuration-plugin:1.6 [INFO] ---------------------------------------------------------------------------- [INFO] Parameter: groupId, Value: unused [INFO] Parameter: artifactId, Value: com.examples.jenkins.plugins [INFO] Parameter: version, Value: 1.0-SNAPSHOT [INFO] Parameter: package, Value: io.jenkins.plugins.sample [INFO] Parameter: packageInPathFormat, Value: io/jenkins/plugins/sample [INFO] Parameter: version, Value: 1.0-SNAPSHOT [INFO] Parameter: package, Value: io.jenkins.plugins.sample [INFO] Parameter: groupId, Value: unused [INFO] Parameter: artifactId, Value: com.examples.jenkins.plugins [INFO] Project created from Archetype in dir: /Users/Shared/CLOUD/Kunde.BSH/workspace/SamplePlugin_Config/com.examples.jenkins.plugins [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 45.525 s [INFO] Finished at: 2020-03-01T17:28:27+01:00 [INFO] ------------------------------------------------------------------------
Verify Plugin
cd com.examples.jenkins.plugins mvn verify
Run Plugin
mvn hpi:run
Working with Groovy Scripts
Include a common groovy script in Jenkins file
1: Create a common.groovy file with function as needed
def mycommoncode() { }
2: In the main Jenkinfile load the file and use the function as shown below
node{ def common = load “common.groovy” common.mycommoncode() }
Basic example of Loading Groovy scripts
File example.groovy
def example1() { println 'Hello from example1' } def example2() { println 'Hello from example2' }
The example.groovy
script defines example1
and example2
functions before ending with return this
. Note that return this
is definitely required and one common mistake is to forget ending the Groovy script with it.Jenkinsfile
def code node('java-agent') { stage('Checkout') { checkout scm } stage('Load') { code = load 'example.groovy' } stage('Execute') { code.example1() } } code.example2()
Processing Github JSON from Groovy
In this demo, we first show how to process JSON response from Github API in Groovy.Processing JSON from Github
String username = System.getenv('GITHUB_USERNAME') String password = System.getenv('GITHUB_PASSWORD') String GITHUB_API = 'https://api.github.com/repos' String repo = 'groovy' String PR_ID = '2' // Pull request ID String url = "${GITHUB_API}/${username}/${repo}/pulls/${PR_ID}" println "Querying ${url}" def text = url.toURL().getText(requestProperties: ['Authorization': "token ${password}"]) def json = new JsonSlurper().parseText(text) def bodyText = json.body // Check if Pull Request body has certain text if ( bodyText.find('Safari') ) { println 'Found Safari user' }
The equivalent bash command for retrieving JSON response from Github API is as follows:Equivalent bash command
// Groovy formatted string String cmd = "curl -s -H \"Authorization: token ${password}\" ${url}" // Example String example = 'curl -s -H "Authorization: token XXX" https://api.github.com/repos/tdongsi/groovy/pulls/2'
Processing Github JSON from Jenkinsfile
Continuing the demo from the last section, we now put the Groovy code into a callable function in a script called “github.groovy”. Then, in our Jenkinsfile, we proceed to load the script and use the function to process JSON response from Github API.github.groovy
import groovy.json.JsonSlurper def getPrBody(String githubUsername, String githubToken, String repo, String id) { String GITHUB_API = 'https://api.github.com/repos' String url = "${GITHUB_API}/${githubUsername}/${repo}/pulls/${id}" println "Querying ${url}" def text = url.toURL().getText(requestProperties: ['Authorization': "token ${githubToken}"]) def json = new JsonSlurper().parseText(text) def bodyText = json.body return bodyText } return this
Jenkinsfile
def code node('java-agent') { stage('Checkout') { checkout scm } stage('Load') { code = load 'github.groovy' } stage('Execute') {
Best Practice
Jenkins | Getting started
Introduction
What is Jenkins. From the Jenkins Homepage, you will get this:
The leading open source automation server, Jenkins provides hundreds of plugins to support building, deploying and automating any project.
As an extensible automation server, Jenkins can be used as a simple CI server or turned into the continuous delivery hub for any project.
This blog will describe, how to setup jenkins and build an environment where we can build and test a simple python app: a calculator.
Setup Jenkins
Jenkins is a java application. So, to run jenkins, we need to things:
- a java development kit
- a jenkins war file
Following the requiements rom the jenkins home page, we should use java8 to run jenkins.
To check your java version, open a console an run
java -version openjdk version "1.8.0_242" OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_242-b08) OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.242-b08, mixed mode)
If you need to install the right java version, get it from here.
Download WAR
Next, we need Jenkins. Download the Jenkins here. We will use the weekly generic java package (.war)
wget http://mirrors.jenkins.io/war/latest/jenkins.war
Create Project Home
We want to keep jenkins and all related files and services in one place.
So, lets create the folder jenkins whereever you want.
First create main folder (we will rever to this as JENKINS_ROOT)
mkdir /home/jenkins
Next, we create the folder JENKINS_HOME. This will the the home directory for the jenkins serice.
mkdir /users/jenkins/home
Remember this folders
export JENKINS_ROOT=/home/jenkins export JENKINS_OME=$JENKINS_ROOT/home
Start Jenkins
java -jar jenkins.war –enable-future-java
Starting Jenkins this way, you will see all log messages on the console.
At this step, the importen messages are the initial admin password:
2020-02-28 16:50:00.749+0000 [id=32] INFO jenkins.install.SetupWizard#init: ************************************************************* ************************************************************* ************************************************************* Jenkins initial setup is required. An admin user has been created and a password generated. Please use the following password to proceed to installation: 6c408145cc964f72ab45cd80e247fa2d This may also be found at: /home/jenkins/home(secrets/initialAdminPassword ************************************************************* ************************************************************* ************************************************************* 2020-02-28 16:50:05.848+0000 [id=57] INFO h.m.DownloadService$Downloadable#load: Obtained the updated data file for hudson.tasks.Maven.MavenInstaller
Start browser and install plugins, create user
$ chrome localhost://8080
Shell access to Jenkins docker
$ docker exec -it jenkins-tutorials bash
Create a Pipeline for a simple Maven Project
Clone the sample maven app
git clone https://github.com/jenkins-docs/simple-java-maven-app
If you want to create a new maven app, you could start with the following command
mvn archetype:generate -DgroupId=com.jcg.maven -DartifactId=HelloWorld -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false cd HelloWorld
Install and Build package
mvm clean install
mvn package
Run your app
java -cp target/MavenHelloWorldProject-1.0-SNAPSHOT.jar com.jcg.maven.App
Troubleshooting
No such DSL method ‘withMaven’ found among steps
This means you don’t have withMaven
as an available DSL method. Most of the time this means you don’t have a plugin installed. In this case, the Pipeline Maven Plugin is required. https://wiki.jenkins.io/display/JENKINS/Pipeline+Maven+Plugin
Links
Local Continuous Delivery Environment With Docker and Jenkins
Additional Readings
Using Oracle Java or OpenJDK.
Install Java on Mac OS with Homebrew
$ brew tap AdoptOpenJDK/openjdk
brew cask install adoptopenjdk8
Install Jenkins with Homebrew on Mac OS
brew install jenkins-lts brew services start jenkins-lts
Install Jenkins with Docker
$ docker run --rm -u root -p 8080:8080 -p 50000:50000 -v jenkins-data:/var/jenkins_home -v /var/run/docker.sock:/var/run/docker.sock -v "$HOME":/home jenkinsci/blueocean