Developer Blog

Tipps und Tricks für Entwickler und IT-Interessierte

Jenkins | Build and Deploy a Groovy App

by Ralph CI/CD, Groovy, Jenkins

Introduction

Using Jenkins as an automation server for your development, you can automate such repeating tasks as testing and deploying your app.

Starting with a sample Groovy App (a simple calculator) with tests, you will learn how to integrate your app in Jenkins and build a pipeline, so that Jenkins runs the desired tasks every time, you change the code.

Prepare the sources

Clone the sample repository from Github.

You should clone the demo repository into you demo account, because you may change some file during this post., and you will not get write permissions for the demo repository.

Also, clone the repository to your local machine to see what our demo app looks like.

$ git clone https://github.com/jenkins-toolbox/SampleApp_GroovyCalculator
Cloning into 'SampleApp_GroovyCalculator'...
remote: Enumerating objects: 194, done.
remote: Counting objects: 100
remote: Compressing objects: 100
remote: Total 194 (delta 44), reused 137 (delta 23), pack-reused 0
Receiving objects: 100
Resolving deltas: 100




Go into the new create folder





$ cd SampleApp_GroovyCalculator/
$ ls
Jenkinsfile      README.md        bin              build.gradle     gradlew          src
Makefile         SampleCalculator build            gradle           settings.gradle
The first task, Jenkins will do in our pipeline: build your app
$ ./gradlew build
Because it’s the first time you start gradlew, the required software will be downloaded:
First: the current Gradle Version (Gradle is the Build Tool used by Groovy Projects)
Downloading https://services.gradle.org/distributions/gradle-6.2.1-bin.zip
………10

Welcome to Gradle 6.2.1!

Here are the highlights of this release:
 - Dependency checksum and signature verification
 - Shareable read-only dependency cache
 - Documentation links in deprecation messages

For more details see https://docs.gradle.org/6.2.1/release-notes.html

Starting a Gradle Daemon, 2 stopped Daemons could not be reused, use --status for details
After this, your app will be tested
> Task :test

Calculator02Spec > two plus two should equal four PASSED

Calculator01Spec > add: 2 + 3 PASSED

Calculator01Spec > subtract: 4 - 3 PASSED

Calculator01Spec > multiply: 2 * 3 PASSED

BUILD SUCCESSFUL in 34s
5 actionable tasks: 5 executed
Perform the build again
No download is required. The build is much quicker.
$ ./gradlew build

BUILD SUCCESSFUL in 1s
5 actionable tasks: 5 up-to-date
Now, test our app:
./gradlew clean test

> Task :test

Calculator02Spec > two plus two should equal four PASSED

Calculator01Spec > add: 2 + 3 PASSED

Calculator01Spec > subtract: 4 - 3 PASSED

Calculator01Spec > multiply: 2 * 3 PASSED

BUILD SUCCESSFUL in 4s
5 actionable tasks: 5 executed
Create a Jenkins Pipeline
Start by clicking on the BlueOcean menu item.
Hint: Blue Ocean is not installed with the default Jenkins installation.
You have to install the corresponding Plugins.
Select Manage Jenkins → Manage Plugins.
Then, select the tab Available and enter in the Filter box: Blue Ocean.
Install all plugins, that will be listed.
Next: Click on the New Pipeline to create your first Pipeline
Use the Item GitHub to specify, where our code is stored
Next, use your GitHub account.
Be sure, that you cloned the demo repository
Next, we select the demo repository SampleApp_GroovyCalculator
Click on Create Pipeline and after a few seconds, the pipeline is created.
Immediately after creating the pipeline, Jenkins is starting the pipeline and all steps included.
If everything went well, you see a positive status
Now, click on the pipeline (e.g. the text master or the status icon) and you will see the pipeline with all steps and their corresponding state.
If you, want to edit the pipeline, for example to add another step, like on the pencil in the header.
Click on Cancel to leave the Pipeline editor.
Hint: If you click on Save, all changes are pushed back to the repository and Jenkins starts the Pipeline again.
Run the Pipeline
If you want to run your pipeline, click on the rerun icon for your pipeline

Jenkins | Cookbook

by Ralph CI/CD, Cookbook, Jenkins

Working with VS Code

Validate Jenkins File

Install VS Code Plugin Jenkins Pipeline Linter Connector

Add configuration in .vscode/settings.json

"jenkins.pipeline.linter.connector.crumbUrl": "<JENKINS_URL>/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,
"jenkins.pipeline.linter.connector.user": "<USERNAME>",
"jenkins.pipeline.linter.connector.pass": "<PASSWORD>",
"jenkins.pipeline.linter.connector.url": "<JENKINS_URL>/pipeline-model-converter/validate",

Replace <USERNAME>, <PASSWORD> and <JENKINS_URL> with your values, for example

"jenkins.pipeline.linter.connector.crumbUrl": "http://localhost:8080/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,%22:%22,//crumb)",
"jenkins.pipeline.linter.connector.user": "admin",
"jenkins.pipeline.linter.connector.pass": "secret",
"jenkins.pipeline.linter.connector.url": "http://localhost:8080/pipeline-model-converter/validate",

Working with Jenkins Client (CLI)

Download Client

wget localhost:8080/jnlpJars/jenkins-cli.jar

Working with Plugins

Create aPlugin

mkdir SamplePlugin
cd SamplePlugin
mvn -U archetype:generate -Dfilter="io.jenkins.archetypes:"
mvn -U archetype:generate -Dfilter="io.jenkins.archetypes:global-configuration-plugin"
[INFO] Scanning for projects...
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-metadata.xml
Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-metadata.xml
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-metadata.xml (14 kB at 32 kB/s)
Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-metadata.xml (20 kB at 44 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-archetype-plugin/maven-metadata.xml
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-archetype-plugin/maven-metadata.xml (918 B at 18 kB/s)
[INFO]
[INFO] ------------------< org.apache.maven:standalone-pom >-------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] >>> maven-archetype-plugin:3.1.2:generate (default-cli) > generate-sources @ standalone-pom >>>
[INFO]
[INFO] <<< maven-archetype-plugin:3.1.2:generate (default-cli) < generate-sources @ standalone-pom <<<
[INFO]
[INFO]
[INFO] --- maven-archetype-plugin:3.1.2:generate (default-cli) @ standalone-pom ---
[INFO] Generating project in Interactive mode
[INFO] No archetype defined. Using maven-archetype-quickstart (org.apache.maven.archetypes:maven-archetype-quickstart:1.0)
Choose archetype:
1: remote -> io.jenkins.archetypes:global-configuration-plugin (Skeleton of a Jenkins plugin with a POM and an example piece of global configuration.)
Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains): : 1
Choose io.jenkins.archetypes:global-configuration-plugin version:
1: 1.2
2: 1.3
3: 1.4
4: 1.5
5: 1.6
Choose a number: 5:
[INFO] Using property: groupId = unused
Define value for property 'artifactId': com.examples.jenkins.plugins
Define value for property 'version' 1.0-SNAPSHOT: :
[INFO] Using property: package = io.jenkins.plugins.sample
Confirm properties configuration:
groupId: unused
artifactId: com.examples.jenkins.plugins
version: 1.0-SNAPSHOT
package: io.jenkins.plugins.sample
 Y: : y
[INFO] ----------------------------------------------------------------------------
[INFO] Using following parameters for creating project from Archetype: global-configuration-plugin:1.6
[INFO] ----------------------------------------------------------------------------
[INFO] Parameter: groupId, Value: unused
[INFO] Parameter: artifactId, Value: com.examples.jenkins.plugins
[INFO] Parameter: version, Value: 1.0-SNAPSHOT
[INFO] Parameter: package, Value: io.jenkins.plugins.sample
[INFO] Parameter: packageInPathFormat, Value: io/jenkins/plugins/sample
[INFO] Parameter: version, Value: 1.0-SNAPSHOT
[INFO] Parameter: package, Value: io.jenkins.plugins.sample
[INFO] Parameter: groupId, Value: unused
[INFO] Parameter: artifactId, Value: com.examples.jenkins.plugins
[INFO] Project created from Archetype in dir: /Users/Shared/CLOUD/Kunde.BSH/workspace/SamplePlugin_Config/com.examples.jenkins.plugins
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  45.525 s
[INFO] Finished at: 2020-03-01T17:28:27+01:00
[INFO] ------------------------------------------------------------------------

Verify Plugin

 cd com.examples.jenkins.plugins
mvn verify

Run Plugin

mvn hpi:run

Working with Groovy Scripts

Include a common groovy script in Jenkins file

1: Create a common.groovy file with function as needed

def mycommoncode() {
}

2: In the main Jenkinfile load the file and use the function as shown below

node{ 
   def common = load “common.groovy”
   common.mycommoncode()
}

Basic example of Loading Groovy scripts

File example.groovy

def example1() {
 println 'Hello from example1' 
}
def example2() {
 println 'Hello from example2'
}

The example.groovy script defines example1 and example2 functions before ending with return this. Note that return this is definitely required and one common mistake is to forget ending the Groovy script with it.Jenkinsfile

def code node('java-agent') { 
    stage('Checkout') { checkout scm } 
    stage('Load') { code = load 'example.groovy' } 
    stage('Execute') { code.example1() }
} 

code.example2()

Processing Github JSON from Groovy

In this demo, we first show how to process JSON response from Github API in Groovy.Processing JSON from Github

String username = System.getenv('GITHUB_USERNAME') 
String password = System.getenv('GITHUB_PASSWORD') 
String GITHUB_API = 'https://api.github.com/repos' String repo = 'groovy' 
String PR_ID = '2' // Pull request ID 
String url = "${GITHUB_API}/${username}/${repo}/pulls/${PR_ID}" 

println "Querying ${url}" 

def text = url.toURL().getText(requestProperties: ['Authorization': "token ${password}"]) 
def json = new JsonSlurper().parseText(text) 

def bodyText = json.body // Check if Pull Request body has certain text if ( bodyText.find('Safari') ) {
     println 'Found Safari user' }

The equivalent bash command for retrieving JSON response from Github API is as follows:Equivalent bash command

// Groovy formatted string 
String cmd = "curl -s -H \"Authorization: token ${password}\" ${url}" 
// Example String 
example = 'curl -s -H "Authorization: token XXX" https://api.github.com/repos/tdongsi/groovy/pulls/2'

Processing Github JSON from Jenkinsfile

Continuing the demo from the last section, we now put the Groovy code into a callable function in a script called “github.groovy”. Then, in our Jenkinsfile, we proceed to load the script and use the function to process JSON response from Github API.github.groovy

import groovy.json.JsonSlurper
def getPrBody(String githubUsername, String githubToken, String repo, String id) {
   String GITHUB_API = 'https://api.github.com/repos' 
   String url = "${GITHUB_API}/${githubUsername}/${repo}/pulls/${id}" 

   println "Querying ${url}"

def text = url.toURL().getText(requestProperties: ['Authorization': "token ${githubToken}"])
def json = new JsonSlurper().parseText(text) 
def bodyText = json.body return bodyText } 
return this

Jenkinsfile

def code node('java-agent') { 
    stage('Checkout') { checkout scm }
    stage('Load') { code = load 'github.groovy' } 
    stage('Execute') {

Best Practice

Jenkins | Getting started

by Ralph CI/CD, Jenkins

Introduction

What is Jenkins. From the Jenkins Homepage, you will get this:

The leading open source automation server, Jenkins provides hundreds of plugins to support building, deploying and automating any project.

As an extensible automation server, Jenkins can be used as a simple CI server or turned into the continuous delivery hub for any project.

This blog will describe, how to setup jenkins and build an environment where we can build and test a simple python app: a calculator.

Setup Jenkins

Jenkins is a java application. So, to run jenkins, we need to things:

  • a java development kit
  • a jenkins war file

Following the requiements rom the jenkins home page, we should use java8 to run jenkins.

To check your java version, open a console an run

java -version
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_242-b08)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.242-b08, mixed mode)

If you need to install the right java version, get it from here.

Download WAR

Next, we need Jenkins. Download the Jenkins here. We will use the weekly generic java package (.war)

wget http://mirrors.jenkins.io/war/latest/jenkins.war

Create Project Home

We want to keep jenkins and all related files and services in one place.

So, lets create the folder jenkins whereever you want.

First create main folder (we will rever to this as JENKINS_ROOT)

mkdir /home/jenkins

Next, we create the folder JENKINS_HOME. This will the the home directory for the jenkins serice.

mkdir /users/jenkins/home

Remember this folders

export JENKINS_ROOT=/home/jenkins
export JENKINS_OME=$JENKINS_ROOT/home

Start Jenkins

java -jar jenkins.war –enable-future-java

Starting Jenkins this way, you will see all log messages on the console.

At this step, the importen messages are the initial admin password:

2020-02-28 16:50:00.749+0000 [id=32]	INFO	jenkins.install.SetupWizard#init:

*************************************************************
*************************************************************
*************************************************************

Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:

6c408145cc964f72ab45cd80e247fa2d

This may also be found at: /home/jenkins/home(secrets/initialAdminPassword

*************************************************************
*************************************************************
*************************************************************

2020-02-28 16:50:05.848+0000 [id=57]	INFO	h.m.DownloadService$Downloadable#load: Obtained the updated data file for hudson.tasks.Maven.MavenInstaller

Start browser and install plugins, create user

$ chrome localhost://8080

Shell access to Jenkins docker

$ docker exec -it jenkins-tutorials bash

Create a Pipeline for a simple Maven Project

Clone the sample maven app

git clone https://github.com/jenkins-docs/simple-java-maven-app

If you want to create a new maven app, you could start with the following command

mvn archetype:generate
        -DgroupId=com.jcg.maven 
        -DartifactId=HelloWorld 
        -DarchetypeArtifactId=maven-archetype-quickstart 
        -DinteractiveMode=false

cd HelloWorld

Install and Build package

mvm clean install

mvn package

Run your app

java -cp target/MavenHelloWorldProject-1.0-SNAPSHOT.jar  com.jcg.maven.App

Troubleshooting

No such DSL method ‘withMaven’ found among steps

This means you don’t have withMaven as an available DSL method. Most of the time this means you don’t have a plugin installed. In this case, the Pipeline Maven Plugin is required. https://wiki.jenkins.io/display/JENKINS/Pipeline+Maven+Plugin

Links

Local Continuous Delivery Environment With Docker and Jenkins

Additional Readings

Using Oracle Java or OpenJDK.

Install Java on Mac OS with Homebrew

$ brew tap AdoptOpenJDK/openjdk
brew cask install adoptopenjdk8

Install Jenkins with Homebrew on Mac OS

brew install jenkins-lts
brew services start jenkins-lts

Install Jenkins with Docker

$ docker run --rm -u root -p 8080:8080  -p 50000:50000 
  -v jenkins-data:/var/jenkins_home 
  -v /var/run/docker.sock:/var/run/docker.sock 
  -v "$HOME":/home 
  jenkinsci/blueocean