Developer Blog

Tipps und Tricks für Entwickler und IT-Interessierte

Jenkins | Build and Deploy a Groovy App

Introduction

Using Jenkins as an automation server for your development, you can automate such repeating tasks as testing and deploying your app.

Starting with a sample Groovy App (a simple calculator) with tests, you will learn how to integrate your app in Jenkins and build a pipeline, so that Jenkins runs the desired tasks every time, you change the code.

Prepare the sources

Clone the sample repository from Github.

You should clone the demo repository into you demo account, because you may change some file during this post., and you will not get write permissions for the demo repository.

Also, clone the repository to your local machine to see what our demo app looks like.

$ git clone https://github.com/jenkins-toolbox/SampleApp_GroovyCalculator
Cloning into 'SampleApp_GroovyCalculator'...
remote: Enumerating objects: 194, done.
remote: Counting objects: 100% (194/194), done.
remote: Compressing objects: 100% (138/138), done.
remote: Total 194 (delta 44), reused 137 (delta 23), pack-reused 0
Receiving objects: 100% (194/194), 93.40 KiB | 817.00 KiB/s, done.
Resolving deltas: 100% (44/44), done.

Go into the new create folder

$ cd SampleApp_GroovyCalculator/
$ ls
Jenkinsfile      README.md        bin              build.gradle     gradlew          src
Makefile         SampleCalculator build            gradle           settings.gradle

The first task, Jenkins will do in our pipeline: build your app

$ ./gradlew build

Because it’s the first time you start gradlew, the required software will be downloaded:

First: the current Gradle Version (Gradle is the Build Tool used by Groovy Projects)

Downloading https://services.gradle.org/distributions/gradle-6.2.1-bin.zip
………10%………20%………30%……….40%………50%………60%……….70%………80%………90%……….100%

Welcome to Gradle 6.2.1!

Here are the highlights of this release:
 - Dependency checksum and signature verification
 - Shareable read-only dependency cache
 - Documentation links in deprecation messages

For more details see https://docs.gradle.org/6.2.1/release-notes.html

Starting a Gradle Daemon, 2 stopped Daemons could not be reused, use --status for details

After this, your app will be tested

> Task :test

Calculator02Spec > two plus two should equal four PASSED

Calculator01Spec > add: 2 + 3 PASSED

Calculator01Spec > subtract: 4 - 3 PASSED

Calculator01Spec > multiply: 2 * 3 PASSED

BUILD SUCCESSFUL in 34s
5 actionable tasks: 5 executed

Perform the build again

No download is required. The build is much quicker.

$ ./gradlew build

BUILD SUCCESSFUL in 1s
5 actionable tasks: 5 up-to-date

Now, test our app:

./gradlew clean test

> Task :test

Calculator02Spec > two plus two should equal four PASSED

Calculator01Spec > add: 2 + 3 PASSED

Calculator01Spec > subtract: 4 - 3 PASSED

Calculator01Spec > multiply: 2 * 3 PASSED

BUILD SUCCESSFUL in 4s
5 actionable tasks: 5 executed

Create a Jenkins Pipeline

Start by clicking on the BlueOcean menu item.

Hint: Blue Ocean is not installed with the default Jenkins installation.

You have to install the corresponding Plugins.

Select Manage JenkinsManage Plugins.

Then, select the tab Available and enter in the Filter box: Blue Ocean.

Install all plugins, that will be listed.

Next: Click on the New Pipeline to create your first Pipeline

Use the Item GitHub to specify, where our code is stored

Next, use your GitHub account.

Be sure, that you cloned the demo repository

Next, we select the demo repository SampleApp_GroovyCalculator

Click on Create Pipeline and after a few seconds, the pipeline is created.

Immediately after creating the pipeline, Jenkins is starting the pipeline and all steps included.

If everything went well, you see a positive status

Now, click on the pipeline (e.g. the text master or the status icon) and you will see the pipeline with all steps and their corresponding state.

If you, want to edit the pipeline, for example to add another step, like on the pencil in the header.

Click on Cancel to leave the Pipeline editor.

Hint: If you click on Save, all changes are pushed back to the repository and Jenkins starts the Pipeline again.

Run the Pipeline

If you want to run your pipeline, click on the rerun icon for your pipeline

Python | Working with Azure

First Step: Hello World Sample

The following steps at borrowed from the quick start tutorial.

Download sample repository

$ git clone https://github.com/Azure-Samples/python-docs-hello-world
$ cd python-docs-hello-world

Create virtual environment

$ python3 -m venv venv
$ source venv/bin/activate
$ pip install -r requirements.txt
$ export FLASK_APP=application.py
$ flask run

Login zu Azure

$ az login

Deploy to App Service

$ az webapp up --sku F1 -n azure-toolbox-flask-demo -l westeurope
webapp azure-toolbox-flask-demo doesn't exist
Creating Resource group 'xx_xx_Linux_westeurope' ...
Resource group creation complete
Creating AppServicePlan 'xx_asp_Linux_westeurope_0' ...
Creating webapp 'flask-demo' ...
Configuring default logging for the app, if not already enabled
Creating zip with contents of dir .../Working-with_Python ...
Getting scm site credentials for zip deployment
Starting zip deployment. This operation can take a while to complete ...
Deployment endpoint responded with status code 202
You can launch the app at http://via-internet-flask-demo.azurewebsites.net
{
  "URL": "http:/azure-toolbox-flask-demo.azurewebsites.net",
  "appserviceplan": "xx_asp_Linux_westeurope_0",
  "location": "westeurope",
  "name": "azure-toolbox--flask-demo",
  "os": "Linux",
  "resourcegroup": "xx_xx_Linux_westeurope",
  "runtime_version": "python|3.7",
  "runtime_version_detected": "-",
  "sku": "FREE",
  "src_path": ".../Working-with_Python"
}

Create Django App with PostgreSQL

Installation PostgreSQL on Mac OS

$ brew install postgres
==> Installing dependencies for postgresql: krb5
==> Installing postgresql dependency: krb5
...
==> Installing postgresql
...
==> Caveats
==> krb5
krb5 is keg-only, which means it was not symlinked into /usr/local, because macOS already provides this software and installing another version in
parallel can cause all kinds of trouble.

If you need to have krb5 first in your PATH run:
  echo 'export PATH="/usr/local/opt/krb5/bin:$PATH"' >> ~/.bash_profile
  echo 'export PATH="/usr/local/opt/krb5/sbin:$PATH"' >> ~/.bash_profile

For compilers to find krb5 you may need to set:
  export LDFLAGS="-L/usr/local/opt/krb5/lib"
  export CPPFLAGS="-I/usr/local/opt/krb5/include"

For pkg-config to find krb5 you may need to set:
  export PKG_CONFIG_PATH="/usr/local/opt/krb5/lib/pkgconfig"

==> postgresql
To migrate existing data from a previous major version of PostgreSQL run:
  brew postgresql-upgrade-database

To have launchd start postgresql now and restart at login:
  brew services start postgresql
Or, if you don't want/need a background service you can just run:
  pg_ctl -D /usr/local/var/postgres start

Set user and passwords for postgres database

Create database and user for django app

$ psql postgres
psql (12.1)
Type "help" for help.

postgres=# CREATE DATABASE pollsdb;
CREATE DATABASE
postgres=# CREATE USER manager WITH PASSWORD '########';
CREATE ROLE
postgres=# GRANT ALL PRIVILEGES ON DATABASE pollsdb TO manager;
GRANT

Download sample repository

$ git clone https://github.com/Azure-Samples/djangoapp.git
$ cd djangoapp

Create virtual environment

$ python3 -m venv venv
$ source venv/bin/activate
$ pip install -r requirements.txt
$ cat env.sh
export DBHOST="localhost"
export DBUSER="manager"
export DBNAME="pollsdb"
export DBPASS="supersecretpass"
$ . env.sh
$ python manage.py  makemigrations
No changes detected
$ python manage.py  migrate
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, polls, sessions
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying polls.0001_initial... OK
  Applying sessions.0001_initial... OK
 $ python manage.py createsuperuser
Username (leave blank to use 'user'): admin
Email address: admin@localhost
Password:
Password (again):
Superuser created successfully.

Run server and acccess web page at http://127.0.0.1:8000/

$ python manage.py runserver
Performing system checks...

System check identified no issues (0 silenced).
January 25, 2020 - 16:42:14
Django version 2.1.2, using settings 'azuresite.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
[25/Jan/2020 16:42:26] "GET / HTTP/1.1" 200 111
[25/Jan/2020 16:42:26] "GET /static/polls/style.css HTTP/1.1" 200 27
Not Found: /favicon.ico
[25/Jan/2020 16:42:26] "GET /favicon.ico HTTP/1.1" 404 2688

Login zu Azure

$ az login

Deploy to App Service

$ az webapp up --sku F1 -n azure-toolbox-django-demo -l westeurope
webapp azure-toolbox-django-demo doesn't exist
Creating Resource group 'xx_xx_Linux_westeurope' ...
Resource group creation complete
Creating AppServicePlan 'xx_asp_Linux_westeurope_0' ...
Creating webapp 'flask-demo' ...
Configuring default logging for the app, if not already enabled
Creating zip with contents of dir .../Working-with_Django ...
Getting scm site credentials for zip deployment
Starting zip deployment. This operation can take a while to complete ...
Deployment endpoint responded with status code 202
You can launch the app at http://via-internet-django-demo.azurewebsites.net
{
  "URL": "http:/azure-toolbox-django-demo.azurewebsites.net",
  "appserviceplan": "xx_asp_Linux_westeurope_0",
  "location": "westeurope",
  "name": "azure-toolbox--django-demo",
  "os": "Linux",
  "resourcegroup": "xx_xx_Linux_westeurope",
  "runtime_version": "python|3.7",
  "runtime_version_detected": "-",
  "sku": "FREE",
  "src_path": ".../Working-with_Django"
}

Additional Reading

Installation

Here is the documentation from Microsoft.

Mac OS

Install with Homebrew

$ brew update && brew install azure-cli
$ az login

ClojureScript | Cookbook

UI: Common tasks

Loading spinner

index.html

<head>
	....
	<link href="lib/bootstrap/4.4.1/css/bootstrap.min.css" rel="stylesheet" type="text/css">
	<link href="lib/css/style.css" rel="stylesheet" type="text/css">
</head>

<body>
	<div id="app">
		<div class="d-flex justify-content-center">
			<div class="loader"></div>
		</div>
	</div>
	<script src="lib/jquery/3.4.1/jquery.min.js" type="text/javascript"></script>
	<script src="lib/bootstrap/4.4.1/js/bootstrap.bundle.min.js" type="text/javascript"></script>
	....

lib/css/style.css

.loader {
	position: fixed;
	z-index: 999;
	overflow: visible;
	margin: auto;

	border: 16px solid #f3f3f3;
	border-radius: 50%;
	border-top: 16px solid blue;

	width: 120px; height: 120px;
	top: 0; left: 0; bottom: 0; right: 0;

	-webkit-animation: spin 2s linear infinite;
	/* Safari */
	animation: spin 2s linear infinite;
}

/* Transparent Overlay */
.loader:before {
	content: '';
	display: block;
	position: fixed;
	top: 0; left: 0;
	width: 100%;
	height: 100%;
}

/* Safari */
@-webkit-keyframes spin {
	0%   { -webkit-transform: rotate(0deg); }
	100% { -webkit-transform: rotate(360deg); }
}

@keyframes spin {
	0%   { transform: rotate(0deg);  }
	100% { transform: rotate(360deg);}
}

Rendering

How to render content

(r/render-component [content]
  (.querySelector js/document "#app"))
(r/render-component [content])
  (.-body js/document)
(defn ^:export run []
  (r/render [simple-example]
  (js/document.getElementById "app"))
(defn home-page []
  [:div
    [:h2 "Homepage!"]])

;; -------------------------
;; Initialize app

(defn mount-root []
  (reagent/render [home-page] (.getElementById js/document "app")))

(defn init! []
  (mount-root))

Render several html tags as one

(defn content-row
    ""
    [col1 col2 col3]
    [:<>
        [:div.grid-item col1]
        [:div.grid-item col2]
        [:div.grid-item col3]])
rxjs

rxjs | Cookbook

Informations

NGRX – Reactive State for Angular

Original Post is here. Github repo has the code snippets at choopage’s GitHub repo.

Recipes


Chaining of observable

The below snippet would return 0, 1, 2, 3…, n until it is stopped. It would be returned at every 2 sec. See reference here.

import { Observable } <strong>from 'rxjs/Rx';

let obs = Observable.<em>interval</em>(2000);
let req = obs
    .flatMap(v => { return Observable.of(v) })
    .subscribe(
        v   => console.log(v),
        err => console.error(err),
        ()  => console.log(<strong>'done'</strong>)
    );

Response transformer

The below would return undefined at every 2 sec interval.

import { Observable } from 'rxjs/Rx';

let obs = Observable.interval(2000);

obs
    .let(responseTransformer)
    .subscribe(
        v  => console.log(v),
        err=> console.error(err)
    );

responseTransformer(input: Observable<any>) {
    return input.map(v => v.data);
}

Using RxJS composition over base class extension

export class MyAPI {
    constructor(private http: Http) {
    }

    get(url: string, options: any) {
        return this.http.get(url, options).let(responseTransformer);
    }
}

responseTransformer(input: Observable<any>) {
    return input.map(v => v.data);
}

Higher Order Observable

We create higher order observable using .map.

const numObservable = Rx.Observable.interval(1000).take(4);
const higherOrderObservable = numObservable.map(x => Rx.Observable.of(1, 2));

higherOrderObservable.subscribe(<br>x => x.subscribe(y => console.log(y)));

Further use of higher order observable

usingHigherOrderObservable() {
    Observable
        .interval(1000)
        .groupBy(n => n
}

Flatten a higher order observable with RxJS switch

const numObservable = Rx.Observable.interval(1000).take(2);
const higherOrderObservable = numObservable.map(x => Rx.Observable.of(1, 2)).switch();

/* 
------+--------+---------
       \        \
        1,2      1,2
        
        switch
        
------1-2-------1-2-------

Switch map flattens the higher order observable
*/

higherOrderObservable.subscribe(x => console.log(x));

switchMap: map and flatten higher order observables

const clickObservable = Rx.Observable.fromEvent(document, 'click');

function performRequest() {
  return fetch('<a href="http://jsonplaceholder.typicode.com/users/1'" target="_blank" rel="noreferrer noopener">http://jsonplaceholder.typicode.com/users/1'</a>)
  .then(res =>; res.json());
  //this returns a Promise
}
//Observabl<Event> --> Observable<Response><br>const responseObservable = clickObservable<br>  .switchMap(click => performRequest());//switchMap can convert Promise to Observable<br>//switchMap = map .... + ... switchresponseObservable.subscribe(x => console.log(x.email));

Use groupBy in real RxJS applications

See reference here.

const busObservable = Rx.Observable.of(
  {code: 'en-us', value: '-TEST-'},
  {code: 'en-us', value: 'hello'},
  {code: 'es', value: '-TEST-'},
  {code: 'en-us', value: 'amazing'},
  {code: 'pt-br', value: '-TEST-'},
  {code: 'pt-br', value: 'olá'},
  {code: 'es', value: 'hola'},
  {code: 'es', value: 'mundo'},
  {code: 'en-us', value: 'world'},
  {code: 'pt-br', value: 'mundo'},
  {code: 'es', value: 'asombroso'},
  {code: 'pt-br', value: 'maravilhoso'}
).concatMap(x => Rx.Observable.of(x).delay(500));const all = busObservable
  .groupBy(obj => obj.code);
  .mergeMap(innerObs => innerObs.skip(1).map(obj => obj.value));//Alternatively could using filter and map
/*
const enUS = busObservable
  .filter(obj => obj.code === 'en-us')
  .map(obj => obj.value);const es = busObservable
  .filter(obj => obj.code === 'es')
  .map(obj => obj.value);const all = Rx.Observable.merge(enUS, es);
*/all.subscribe(x => console.log(x));

Using .map versus .switchMap

The below code snippet we can view the result of using .map versus .switchMap

//user.service.ts
import { Injectable } from '@angular/core';
import { Http } from '@angular/http';
import { Observable } from "rxjs";
import 'rxjs/add/operator/map';
import 'rxjs/add/operator/do';
import 'rxjs/add/operator/switchMap';
import 'rxjs/add/operator/filter';
import 'rxjs/add/operator/catch';
import 'rxjs/add/operator/debounceTime';
import 'rxjs/add/operator/distinctUntilChanged';

@Injectable()
export class UserService {

  constructor(private http: Http) { }

  getUsers(): Observable<any> {
    return this.http.get('http://jsonplaceholder.typicode.com/users')
        //.map(v => v.json());
        .switchMap(v => v.json());
  }

}
//app.component.ts
import { Component } from '@angular/core';
import { UserService } from "./user.service";

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css']
})
export class AppComponent {
  title = 'app works!';

  constructor(private userService: UserService) {

  }

  search(term: string) {
    this.userService.getUsers()
        .subscribe(v => console.log(v));

    /*
    //we can do this with .switchMap
    this.userService.getUsers()
        .subscribe(v => {if (v.email != "Sincere@april.biz") {
          console.log(v.email);
        }});
     */
  }
}

Solving the multiple Async Pipe in Angular ≥ 2.0.0 with share operator

Remember to import import "rxjs/add/operator/share"; See reference here.

squareData$: Observable<string> = Observable.range(0, 10)
        .map(x => x * x)
        .do(x => console.log(`CalculationResult: ${x}`)
        .toArray()
        .map(squares => squares.join(", "))
        .share();  // remove this line: console will log every result 3 times instead of 1

Managing Cold and Hot Observables using publish().refCount() which is similar to .share()

ngOnInit() {
    // in angular 2 and above component.ts file add these    this.coldObservable();
    this.hotObservable();
}
/*
* cold observable is like a recast of video
* */
coldObservable() {
    let incrementalObs = Observable.interval(1000).take(10).map(x => x + 1);
    incrementalObs.subscribe(val => console.log('a: ' + val));
    setTimeout(function() {
        incrementalObs.subscribe(val => console.log('      b: ' + val));
    }, 4500);
}


/*
* hot observable is like watching a live video
* */
hotObservable() {
    let incrementalObs = Observable.interval(1000).take(10).map(x => x + 1).publish().refCount(); //can also use .share()
    incrementalObs.subscribe(val => console.log('a: ' + val));
    setTimeout(function() {
        incrementalObs.subscribe(val => console.log('      b: ' + val));
    }, 4500);
}

Observables Array Operations with flatMap

Reference Rangle.io article.

getLoadList(): void {
  this.shareService
      .fetchLoad()
      .take(1)
      .filter(response => {
        if ( response.status === 200 ) {
          return true;
        } else if ( response.status === 304 ) {
          // do something more
          return false;
        } else {
          this.gotoErrorPage();
          return false;
        }
      })
      .flatMap(response => response.data.loads as Load[])
      .filter(obj => obj.content.contentGrade === 'x')
      .subscribe(
          val => console.log(val),
          err => {
            console.error(err);
          });
}

Error Handling


Error handling in RxJS

Some learning points from RxJS lesson videos. This repo is available in my GitHub repo.

import { Component, OnInit } from '@angular/core';

import { Observable } from 'rxjs/Observable';
import { Subject } from 'rxjs/Subject';
import { BehaviorSubject } from "rxjs/BehaviorSubject";

import 'rxjs/add/observable/bindCallback';
import 'rxjs/add/observable/bindNodeCallback';
import 'rxjs/add/observable/combineLatest';
import 'rxjs/add/observable/concat';
import 'rxjs/add/observable/defer';
import 'rxjs/add/observable/empty';
import 'rxjs/add/observable/forkJoin';
import 'rxjs/add/observable/from';
import 'rxjs/add/observable/fromEvent';
import 'rxjs/add/observable/fromEventPattern';
import 'rxjs/add/observable/fromPromise';
import 'rxjs/add/observable/generate';
import 'rxjs/add/observable/if';
import 'rxjs/add/observable/interval';
import 'rxjs/add/observable/merge';
import 'rxjs/add/observable/race';
import 'rxjs/add/observable/never';
import 'rxjs/add/observable/of';
import 'rxjs/add/observable/onErrorResumeNext';
import 'rxjs/add/observable/pairs';
import 'rxjs/add/observable/range';
import 'rxjs/add/observable/using';
import 'rxjs/add/observable/throw';
import 'rxjs/add/observable/timer';
import 'rxjs/add/observable/zip';
import 'rxjs/add/observable/dom/ajax';
import 'rxjs/add/observable/dom/webSocket';
import 'rxjs/add/operator/buffer';
import 'rxjs/add/operator/bufferCount';
import 'rxjs/add/operator/bufferTime';
import 'rxjs/add/operator/bufferToggle';
import 'rxjs/add/operator/bufferWhen';
import 'rxjs/add/operator/catch';
import 'rxjs/add/operator/combineAll';
import 'rxjs/add/operator/combineLatest';
import 'rxjs/add/operator/concat';
import 'rxjs/add/operator/concatAll';
import 'rxjs/add/operator/concatMap';
import 'rxjs/add/operator/concatMapTo';
import 'rxjs/add/operator/count';
import 'rxjs/add/operator/dematerialize';
import 'rxjs/add/operator/debounce';
import 'rxjs/add/operator/debounceTime';
import 'rxjs/add/operator/defaultIfEmpty';
import 'rxjs/add/operator/delay';
import 'rxjs/add/operator/delayWhen';
import 'rxjs/add/operator/distinct';
import 'rxjs/add/operator/distinctUntilChanged';
import 'rxjs/add/operator/distinctUntilKeyChanged';
import 'rxjs/add/operator/do';
import 'rxjs/add/operator/exhaust';
import 'rxjs/add/operator/exhaustMap';
import 'rxjs/add/operator/expand';
import 'rxjs/add/operator/elementAt';
import 'rxjs/add/operator/filter';
import 'rxjs/add/operator/finally';
import 'rxjs/add/operator/find';
import 'rxjs/add/operator/findIndex';
import 'rxjs/add/operator/first';
import 'rxjs/add/operator/groupBy';
import 'rxjs/add/operator/ignoreElements';
import 'rxjs/add/operator/isEmpty';
import 'rxjs/add/operator/audit';
import 'rxjs/add/operator/auditTime';
import 'rxjs/add/operator/last';
import 'rxjs/add/operator/let';
import 'rxjs/add/operator/every';
import 'rxjs/add/operator/map';
import 'rxjs/add/operator/mapTo';
import 'rxjs/add/operator/materialize';
import 'rxjs/add/operator/max';
import 'rxjs/add/operator/merge';
import 'rxjs/add/operator/mergeAll';
import 'rxjs/add/operator/mergeMap';
import 'rxjs/add/operator/mergeMapTo';
import 'rxjs/add/operator/mergeScan';
import 'rxjs/add/operator/min';
import 'rxjs/add/operator/multicast';
import 'rxjs/add/operator/observeOn';
import 'rxjs/add/operator/onErrorResumeNext';
import 'rxjs/add/operator/pairwise';
import 'rxjs/add/operator/partition';
import 'rxjs/add/operator/pluck';
import 'rxjs/add/operator/publish';
import 'rxjs/add/operator/publishBehavior';
import 'rxjs/add/operator/publishReplay';
import 'rxjs/add/operator/publishLast';
import 'rxjs/add/operator/race';
import 'rxjs/add/operator/reduce';
import 'rxjs/add/operator/repeat';
import 'rxjs/add/operator/repeatWhen';
import 'rxjs/add/operator/retry';
import 'rxjs/add/operator/retryWhen';
import 'rxjs/add/operator/sample';
import 'rxjs/add/operator/sampleTime';
import 'rxjs/add/operator/scan';
import 'rxjs/add/operator/sequenceEqual';
import 'rxjs/add/operator/share';
import 'rxjs/add/operator/single';
import 'rxjs/add/operator/skip';
import 'rxjs/add/operator/skipUntil';
import 'rxjs/add/operator/skipWhile';
import 'rxjs/add/operator/startWith';
import 'rxjs/add/operator/subscribeOn';
import 'rxjs/add/operator/switch';
import 'rxjs/add/operator/switchMap';
import 'rxjs/add/operator/switchMapTo';
import 'rxjs/add/operator/take';
import 'rxjs/add/operator/takeLast';
import 'rxjs/add/operator/takeUntil';
import 'rxjs/add/operator/takeWhile';
import 'rxjs/add/operator/throttle';
import 'rxjs/add/operator/throttleTime';
import 'rxjs/add/operator/timeInterval';
import 'rxjs/add/operator/timeout';
import 'rxjs/add/operator/timeoutWith';
import 'rxjs/add/operator/timestamp';
import 'rxjs/add/operator/toArray';
import 'rxjs/add/operator/toPromise';
import 'rxjs/add/operator/window';
import 'rxjs/add/operator/windowCount';
import 'rxjs/add/operator/windowTime';
import 'rxjs/add/operator/windowToggle';
import 'rxjs/add/operator/windowWhen';
import 'rxjs/add/operator/withLatestFrom';
import 'rxjs/add/operator/zip';
import 'rxjs/add/operator/zipAll';
@Component({
    selector: 'app-root',
    templateUrl: './app.component.html',
    styleUrls: [ './app.component.css' ]
})
export class AppComponent implements OnInit {
    title = 'app works!';
    obs = Observable.of(1, 2, 3, 4);

    ngOnInit() {
        this.howToHandleErrorV1();
        this.howToHandleErrorV2();
        this.howToUseRetry();
        this.mergeObservableAndThrowError();
        this.mergeObservableAndErrorResumeNext();
        this.mergeObservableAndErrorCatch();    
    }

    
    
    /*
     * This uses Catch for V1. This introduces Closure. It is effectively the same as V2.
     * */
    howToHandleErrorV1() {
        this.obs
            .map(x => {
                if ( x === 3 ) {
                    throw 'I hate threes'; // When it hitted error it actually unsubscribe itself at x === 3 of throw error
                }
                return x;
            })
            .catch(err => Observable.throw('Caught error here Observable.throw')) // continue go down the error path use Observable.throw
            .catch(err => Observable.of('Caught error here Observable.of')) // catch just use Observable.of
            .subscribe(
                x => console.log(x),
                err => console.error(err), // If not catch any where, the I hate threes errors will be propagated to here
                () => console.log('done completed')
            );
    }

    
    
    /*
     * There is a difference between V1 and V2. For V2 it is using onErrorResumeNext which
     * */
    howToHandleErrorV2() {
        let good = Observable.of('Caught error here Observable.of');

        this.obs
            .map(x => {
                if ( x === 3 ) {
                    throw 'I hate threes'; // When it hit error it actually unsubscribe itself at x === 3 of throw error
                }
                return x;
            })
            .onErrorResumeNext(good) // To catch just use Observable.of
            .subscribe(
                x => console.log(x),
                err => console.error(err), // If not catch any where, the I hate threes errors will be propagated to here
                () => console.log('done completed')
            );
    }

    
    
    /*
     * For this we use see it retries three times then console.error(err);
     * So retryWhen is for trying network connection websocket
     * */
    howToUseRetry() {
        this.obs
            .map(x => {
                if ( x === 3 ) {
                    throw 'I hate threes'; // When it hitted error it actually unsubscribe itself at x === 3 of throw error
                }
                return x;
            })
            .retry(3) // retry three times
            .retryWhen(err => err.delay(2000).take(3)) // similar but with 2 seconds delay and the error is not propagated.
            .retryWhen(err => err.delay(2000).take(3).concat(Observable.throw('bad'))) // this it would throw an error.
            .subscribe(
                x => console.log(x),
                err => console.error(err), // If not catch any where, the I hate threes errors will be propagated to here
                () => console.log('done completed')
            );
    }
/*
 * Using observable merge operator
 * */
mergeObservableAndThrowError() {
    let mergedObs = Observable.merge(
        this.obs, //1, 2, 3, 4
        Observable.throw('Stop Error'),
        Observable.from(this.array), //0, 1, 2, 3, 4, 5
        Observable.of(999) //999,
    );

    mergedObs.subscribe(
        val => console.log(val), //this should show 1, 2, 3, 4, Stop Error
        error => console.log(error),
        () => console.log("completed")
    );
}

/* Using observable onErrorResumeNext just like merge operator
 * */
mergeObservableAndErrorResumeNext() {
    let mergedObs = Observable.onErrorResumeNext(
        this.obs, //1, 2, 3, 4
        Observable.throw('Stop Error'),
        Observable.from(this.array), //0, 1, 2, 3, 4, 5
        Observable.of(999) //999,
    );

    mergedObs.subscribe(
        val => console.log(val), //this should show 1, 2, 3, 4, 0, 1, 2, 3, 4, 5, 999
        error => console.log(error),
        () => console.log("completed")
    );
}
/*
 * Using observable merge operator and catch
 * */
mergeObservableAndErrorCatch() {
    let mergedObs = Observable.merge(
        this.obs, //1, 2, 3, 4
        Observable.throw('Stop Error'),
        Observable.from(this.array), //0, 1, 2, 3, 4, 5
        Observable.of(999) //999,
    ).catch(e => {
        console.log(e);
        return Observable.of('catch error here');
    });

    mergedObs.subscribe(
        val => console.log(val), //this should show 1, 2, 3, 4, Stop Error, Catch Error Here
        error => console.log(error),
        () => console.log("completed")
    );
}
}

map vs flatMap in RxJS

Transform the items emitted by an Observable into Observables, then flatten the emissions from those into a single Observable. See my GitHub repo.

obs = Observable.of(1, 2, 3, 4);

ngOnInit() {
    this.usingMap();
    this.usingMapToMakeInnerObservable();
    this.usingMapAndMergeAll();
    this.usingFlatMap();
}usingMap() {
    this.obs
        .map(x => x * 2) // transform the input by multiple of 2
        .subscribe(
            x => console.log(x),
            err => console.error(err),
            () => console.log('done completed')
        );
}
usingMapToMakeInnerObservable() {
    this.obs
        .map(x => Observable.timer(500).map(() => x + 3)) // transform the input wrapping it with another observable and addition of 3
        //.map(x => Observable.timer(500).map((x) => x + 3)) // !!! REMEMBER Not the same as the immediate above
        .subscribe(
            x => console.log(x),
            err => console.error(err),
            () => console.log('done completed')
        );
}
// Map and Merge all is the same as just one FlatMap
usingMapAndMergeAll() {
    this.obs
        .map(x => Observable.timer(500).map(() => x + 3)) // transform the input wrapping it with another observable and addition of 3
        .mergeAll()
        .subscribe(
            x => console.log(x),
            err => console.error(err),
            () => console.log('done completed')
        );
}
// Flat map is the same as map then merge all
// transform the items emitted by an Observable into Observables, then flatten the emissions from those into a single Observable
usingFlatMap() {
    this.obs
        .flatMap(x => Observable.timer(500).map(() => x + 10)) // transform the input wrapping it with another observable and addition of 10
        .subscribe(
            x => console.log(x),
            err => console.error(err),
            () => console.log('done completed')
        );
}

Transforming pure Javascript array vs. Observable from array

See GitHub for source code

array = [0, 1, 2, 3, 4, 5];

ngOnInit() {
    this.setArrayToObservableThenTransform();
}/*
* This keeps creating new array. It is good that it creates new array of arr for immutability.
* But it's bad because there is clean up and resource intensive for mobile
* */
transformArray() {
    let result = this.array
        .filter(( x, i, arr ) => {
            console.log('filtering ' + x);
            console.log('is the source array ' + (arr === this.array));
            return x
        })
        .map(( x, i, arr ) => {
            console.log('mapping ' + x);
            console.log('is the source array ' + (arr === this.array));
            return x + '!';
        })
        .reduce(( r, x, i, arr ) => {
            console.log('reducing ' + x);
            return r + x;
        }, '--');

    console.log(result);
}

/*
* This is more efficient for resource management because it linearly scans and discard when not right
* */
setArrayToObservableThenTransform() {    let obsArray = Observable.from(this.array); // Use Observable.from() instead of Observable.of(). There is diff.    obsArray
        .filter(( x: any ) => {
            console.log('filtering ' + x);
            return x
        })
        .map(( x ) => {
            console.log('mapping ' + x);
            return x + '!';
        })
        .reduce(( r, x ) => {
            console.log('reducing ' + x);
            return r + x;
        }, '--')
        .subscribe(
            x => console.log(x)
        );
}

Using reduce and scan to aggregate RxJs data

See GitHub for source code

array = [0, 1, 2, 3, 4, 5];ngOnInit() {
    this.reduceArray();
    this.reduceObservableArray();
    this.reduceObservableArray_Abstract2();
    this.scanObservableArray();
}
/*
* This is the same as reduceObservableArray()
* */
reduceArray() {
    let result = this.array.reduce(
        (accumulator, currentValue) => accumulator + currentValue, 3
    ); // 3 is the init value.
    console.log('reduceArray ' + result); // output 18 => 3 + (0 ... 5)
}
/*
* This is the same as reduceArray()
* But this waits for all the arrays to finish emitting before reducing them to one single number
* See the next method to understand better
* */
reduceObservableArray() {
    let obsArray = Observable.from(this.array);
    obsArray.reduce(
        (accumulator, currentValue) => accumulator + currentValue, 3
    ).subscribe(
        val => console.log('reduceObservableArray ' + val)
    );
}
/*
* The exact same reduce function/method as of reduceObserableArray() above
* This proves that it waits for all 6 numbers to come in then reduce them
* */
reduceObservableArray_Abstract2() {
    let obsArray = Observable.interval(1000).take(6); //emits 6 times of 0, 1, 2, 3, 4, 5
    obsArray.reduce(
        (accumulator, currentValue) => accumulator + currentValue, 3
    ).subscribe(
        val => console.log('reduceObservableArray_Abstract2 ' + val)
    );
}
/*
* This is the same as the above reduceObserableArray_Abstract2()
* except this is using scan instead of reduce
* */
scanObservableArray() {
    let obsArray = Observable.interval(1000).take(6); //emits 6 times of 0, 1, 2, 3, 4, 5
    obsArray.scan(
        (accumulator, currentValue) => accumulator + currentValue, 3
    ).subscribe(
        val => console.log('scanObservableArray() ' + val)
    );
}

Create, next, and subscribe to Subject and BehaviorSubject

There is a Stack Overflow thread which discussed about the difference between Subject and BehaviorSubject. It’s worth understanding.

Also see my personal GitHub for source code.

app.component.ts

import { Subject } from 'rxjs/Subject';
import { BehaviorSubject } from "rxjs/BehaviorSubject";
// create subject
// there is no need for initial value
subject = new Subject<boolean>();

// create behaviorSubject which require initial value
// true is an initial value. if there is a subscription
// after this, it would get true value immediately
behaviorSubject = new BehaviorSubject<boolean>(true);
ngOnInit() {
    this.subject.next(false); /* Subject subscription wont get anything at this point before the subscribeSubject() */    this.subscribeSubject();
    this.subscribeBehaviorSubject();
}
/*
* Push the next val into the behavior subject
* */
nextSubject(val: boolean) {
    this.subject.next(val);
}

/*
* Any values push into the subject would not be can shown
* before this subscribeSubject() is called
* */
subscribeSubject() {
    this.subject
        //.take(1) //when we include .take(1) we will have a complete. Without this it will continue subscribing
        .subscribe(
            val => console.log(val),
            err => console.error(err),
            () => console.log('completed')
        );
}

/*
* This is the proper way to return a subject as observable
* */
getSubject(): Observable<boolean> {
    return this.subject.asObservable();
}

/*
 * Push the next val into the behavior subject
 * */
nextBehaviorSubject(val: boolean) {
    this.behaviorSubject.next(val);
}

/*
* For angular Behavior subject for a data service as a angular service often initializes
* before component and behavior subject ensures that the component consuming the
* service receives the last updated data even if there are no new
* updates since the component's subscription to this data.
* */
subscribeBehaviorSubject() {
    this.behaviorSubject
        // .first()
        .subscribe(
            val => console.log(val),
            err => console.error(err),
            () => console.log('completed')
        );
}

app.component.html

Subject:
<button (click)="nextSubject(true)">true</button>
<button (click)="nextSubject(false)">false</button>

<br>
BehaviorSubject:
<button (click)="nextBehaviorSubject(true)">true</button>
<button (click)="nextBehaviorSubject(false)">false</button>

Finally operator

usingFinallyOperator() {
    Observable
        .interval(500)
        .take(4)
        .finally(() => console.log('End of the observable, Hello World'))
        .subscribe(
            val => console.log('count taker ' + val)
        );
}

Stopping / Intercepting Observable

Imagine using Gmail where it allows you to undo email sent? We can produce similar experience with Observable

// subscription is created when an observable is being subscribed
subscription: Subscription;

// boolean variable for showing stop observable using takeWhile operator
isTrue: boolean = true;
/*
* basic interval can be used as delay too
* Imagine Gmail allows you to send and undo send within 4 seconds of sending
* Use Case: Perform an action 8 seconds later then intercept if user choose to undo the action
* */basicInterval() {
    let undoInSeconds: number = 8;
    this.subscription = Observable
            .interval(1000)
            .take(undoInSeconds)
            .takeWhile(() => this.isTrue)
            .subscribe(
                (val: number) => {
                    console.log(`${val + 1} seconds...         UNDO`);
                    ( val === (undoInSeconds - 1) ) ? console.log('Email sent / Action performed') : null;
                }
            );
}
/*
* This is to stop observable from continuing performance
* Use Case: Stop observable from running like how Gmail could undo email being sent
* */stopObservableUsingUnsubscribe() {
    if (!!this.subscription) {
        this.subscription.unsubscribe();
        console.log('subscription: Subscription is unsubscribed');
    }
}

/*
* This is also to stop observable from continuing performance
* This method is more preferable than subscribing method then unsubscribe
 * Use Case: Stop observable from running like how Gmail could undo email being sent
* */stopObservableUsingTakeWhile() {
    this.isTrue = false;
}

Perform conditional Reactive Form validation

This is my approach to performing conditional validation when using Angular. We will minimally manipulate the Observable of RxJS in this example. Let’s try by creating or using app.component.ts.

1 . Create a form that has two form controls reason and otherReason.

/* * Refer to angular official guide at https://angular.io/guide/reactive-forms on how to create reactive form with form controls * */createForm() {
    this.form = this.formBuilder.group({      reason: ['', Validators.required ],
      otherReason: [''],    });
}

2 . Create two methods addValidator and removeValidator.

/*  * For conditional form validation use  * */
private addValidator( control: AbstractControl, newValidator ){
    let existingValidators = control.validator;
    control.setValidators(Validators.compose([ existingValidators, newValidator ]));
    control.updateValueAndValidity();
}

/*  * For conditional form validation use  * */
private removeValidator( control: AbstractControl ){
    control.clearValidators();
    control.updateValueAndValidity();
}

3 . Create third method called conditionalFormValidation.

conditionalFormValidation( parentField, childField, matchValue = 'Others' )
{
    this.form
        .get(parentField)
        .valueChanges
        .forEach(( value: string ) => {
            const childFieldControl: AbstractControl = this.form.get(childField);
            if ( value === matchValue ) {
                this.addValidator(childFieldControl, Validators.required);
            } else {
                this.removeValidator(childFieldControl);
            }
        });
}

4 . Create or add to ngOnInit method

ngOnInit() {    
    this.createForm();
    this.conditionalFormValidation('reason', 'otherReason');
}

The outcome should illustrate that if we select ‘Others’ option in the dropdown list of reason, it should make otherReason form control field as required.

Drop-down selection for reason e.g. reason for absent from work
On selecting Others, the conditional form validation would make otherReason field turn into a required field

Perform manual operations after reading from Firebase database

When using AngularFire2 with Angular + Firebase, in getting a list of data from Firebase, we will get one observable instance but within that one observable is an array of N size. We can manually filter the array inside that one observable instance using arr.filter. It is different from RxJS .filter operator. Of course we can also flatten what is inside an array using .flatMap() operator. However, we’re going use JavaScript array filtering, instead of non-observable filtering, right after getting an observable object.

this.db.list(`/review`)
       .map(arr => arr.filter(item => item.rating > 3));

We can also reverse an array using JavaScript array reverse function. See reference. On a side note, using negative timestamp to reverse Firebase display is also another option.

this.db.list('/review')
       .map(arr => { return arr.reverse(); });

Besides those above, we can also use the response returned from AngularFire2 to perform “local filter/search”. This result can be valuable for autocomplete filtered list or searches. However, this approach below suffers severely in performance issue where at the magnitude of the size of the returned response from AngularFire2. E.g. if the list has N items. It has to iterate at least 1N. Perhaps an average of 2N.

getFilteredClientList(searchQuery: string): Observable<Client[]> {
    let query = searchQuery.trim().toLowerCase();

    return this.db.list(`/client`)
      .map((arr: Client[]) => {
        return arr.filter((item: Client) => {
          return item.email.toLowerCase().indexOf(query) === 0
            || item.given_name.toLowerCase().indexOf(query) === 0
            || item.family_name.toLowerCase().indexOf(query) === 0
        });
      });
  }
Test-Driven Development with Python

Python | Test-Driven Development

  • Part 1: Create a TDD Python Project
  • Part 2: Use Jenkins to automatically test your App

Part 1: Create a TDD Python Project

Final source code is on Github.

Introduction

The task of creating an error free program is not easy. And, if your program runs free of errors, keeping it error-free after an update or change is even more complicated. You don’t want to insert new errors or change correct code with wrong parts.

The answer to this situation (directly from the Oracle of Delphi) is: Testing, Testing, Testing

And the best way to test is to start with tests.

This means: think about what the result should be and then create a Test that checks this. Imagine, you have to write a function for adding two values, and you should describe the functionality.

So, maybe, your description contains one or two examples:

My functions add’s two numbers, e.g 5 plus 7 is 12 (or at least should be 12 :))

The procedure with the TDD is:

  • think and define, what the function should to
  • write a stub for the function, e.g. only function parameters and return type
  • write a function, that tests you function with defines parameters and know result

For our example above, this means:

Write the python script with the desired functionality: src/main.py

def add(val1,val2):
    return 0 # this is only a dummy return value

Write the Python Testscript: tst/main.p

def_test_add():
    result = add(5,7)

    if (result = 12):
        print("everything fine")
    else:
        printf("ups, problems with base arithmetics")

Now, with these in your toolbox, you can always verify your code by running the tests.

$ python test_add.py
ups, problems with base arithmetics

dfdf

Setup virtual environment

Mostly, tests are repeated after every change. So, to be sure, that each test is running the same way and with the same environment, we will use pythons virtual environment feature to create a new fresh python environment for the tests.

Create virtual environment

$ python3 -m venv .env/python

Activate environment

Add the following line to .bashrc (or .envrc if you are using direnv)

$ . .env/python/bin/activate

Install required packages

$ pip install pytest

Create a sample Application

Prepare folder

Create folder for sources

$ mkdir src

Create sample package

$ mkdir src/CalculatorLib
$ touch src/CalculatorLib/__init__.py
$ touch src/CalculatorLib/Calculator.py

At least, create a simple Calculator: src/CalculatorLib/Calculator.py

class Calculator:
    def __init__(self):
        print("Init Calculator")

    def add(self, a, b):
        return a + b

    def subtract(self, a, b):
        return a - b

    def multiply(self, a, b):
        return a * b

    def divide(self, a, b):
        return a / b

    def power(self, base, exp):
        return base ** exp

Create the Main App for your Calculator: src/main.py

from CalculatorLib.Calculator import Calculator

class Main(object):

    def run(self):
        c = Calculator()

        print("5 + 3 =
        print("8 - 4 =
        print("5 * 3 =
        print("8 / 4 =

        print("8 ^ 4 =

if __name__ == '__main__':
    Main().run()

Yur done with the fist development step. Try your app:

$ python src/main.py
Init Calculator
5 + 3 =     8
8 - 4 =     4
5 * 3 =    15
8 / 4 =     2
8 ^ 4 =  4096

Add Unit Tests

We will start with our first test. Create folder for tests and a file tst/main.py

$ mkdir tst
$ touch tst/main.py

Use the following for your test script tst/main.py

from CalculatorLib.Calculator import Calculator
import unittest

class CalculatorTest(unittest.TestCase):

    @classmethod
    def setUpClass(self):
        self.c = Calculator()

    def test_add(self):
        self.assertEqual(8, self.c.add(5, 3))

    def test_subtract(self):
        self.assertEqual(4, self.c.subtract(8, 4))

    def test_multiply(self):
        self.assertEqual(32, self.c.multiply(8, 4))

    def test_divide(self):
        self.assertEqual(2, self.c.divide(8, 4))
            
    def test_power(self):
        self.assertEqual(16, self.c.power(2, 4))
                                    
if __name__ == '__main__':
    unittest.main()

Finally try your test script:

$ PYTHONPATH=./src python -m pytest tst/main.py  --verbose
================================= test session starts ================================
platform darwin -- Python 3.7.4, pytest-4.4.1, py-1.8.0, pluggy-0.9.0 -- <Testproject_Python-Calculator/.env/python/bin/python>
cachedir: .pytest_cache
rootdir: <Testproject_Python-Calculator>
plugins: cov-2.6.1
collected 5 items

tst/main.py::CalculatorTest::test_add PASSED             [ 20%]
tst/main.py::CalculatorTest::test_divide PASSED          [ 40%]
tst/main.py::CalculatorTest::test_multiply PASSED        [ 60%]
tst/main.py::CalculatorTest::test_power PASSED           [ 80%]
tst/main.py::CalculatorTest::test_subtract PASSED        [100%]

The command to run the test is python -m pytest tst/main.py, but why the lead Variable PYTHONPATH?

Try it without:

$ python -m pytest tst/main.py
=================================== test session starts ==================================
platform darwin -- Python 3.7.4, pytest-4.4.1, py-1.8.0, pluggy-0.9.0 -- ##/Testproject_Python-Calculator/.env/python/bin/python
cachedir: .pytest_cache
rootdir: ##/Testproject_Python-Calculator
plugins: cov-2.6.1
collected 0 items / 1 errors

========================================= ERRORS =========================================
____________________________________ ERROR collecting tst/main.py ________________________
ImportError while importing test module '##/Testproject_Python-Calculator/tst/main.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tst/main.py:2: in <module>
    from CalculatorLib.Calculator import Calculator
E   ModuleNotFoundError: No module named 'CalculatorLib'
!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!
================================== 1 error in 1.84 secon==================================

Recognize the ModuleNotFoundError in line 16! This means, that Python could not find the desired CalculatorLib.

Look at your folder structure:

$ tree .
.
├── src
│   ├── CalculatorLib
│   │   ├── Calculator.py
│   │   ├── init__.py
│   └── main.py
└── tst
    └── main.py

.

In your Testscript, we import the CalculatorLib whit this statement:

from CalculatorLib.Calculator import Calculator

Python is interpreting this in the following way:

  • Look in the folder of the test script for a subfolder with the name CalculatorLib
  • There, look for a file Calculator.py
  • And in this file, use the class Calculator

Obviously, the folder CalculatorLib is NOT in the same folder as the test script: it is part of the src folder.

So, using the environment variable PYTHONPATH, we inform python where to search python scripts and folders.

Add additional functionality

Add a function at the end of your Calculator: src/CalculatorLib/Calculator.py

    ....
    def factorial(self, n):
        return 0

Add a call of the new function to your main app: src/main.py

    ...
    def run(self):
        ...
        print("4!    =



Add a test for the new function to your test script: tst/main.py

    ...
    def test_factorial(self):
        self.assertEqual(24, self.c.factorial(4))

Try it:

$ python src/main.py
Init Calculator
5 + 3 =     8
8 - 4 =     4
5 * 3 =    15
8 / 4 =     2
8 ^ 4 =  4096
$ PYTHONPATH=./src python -m pytest tst/main.py
==================================== test session starts =====================================
platform darwin -- Python 3.7.4, pytest-4.4.1, py-1.8.0, pluggy-0.9.0
rootdir: ##/Testproject_Python-Calculator
plugins: cov-2.6.1
collected 6 items

tst/main.py ..F...                                                                      [100%]

========================================== FAILURES ==========================================
_______________________________ CalculatorTest.test_factorial ________________________________

self = <main.CalculatorTest testMethod=test_factorial>

    def test_factorial(self):
>       self.assertEqual(24, self.c.factorial(4))
E       AssertionError: 24 != 0

tst/main.py:31: AssertionError
============================= 1 failed, 5 passed in 0.14 seconds =============================

Test failed, was we expect it.

Now, implement the function correctly and startover the test:

Add a function at the end of your Calculator: src/CalculatorLib/Calculator.py

import math

class Calculator:
    ...
    def factorial(self, n):
       if not n >= 0:
            raise ValueError("n must be >= 0")

        if math.floor(n) != n:
            raise ValueError("n must be exact integer")

        if n+1 == n:  # catch a value like 1e300
            raise OverflowError("n too large")

        result, factor = 1, 2
        
        while factor <= n:
            result *= factor
            factor += 1

        return result
$ PYTHONPATH=./src python -m pytest tst/main.py  --verbose
==================================== test session starts =====================================
platform darwin -- Python 3.7.4, pytest-4.4.1, py-1.8.0, pluggy-0.9.0 -- ##/Testproject_Python-Calculator/.env/python/bin/python
cachedir: .pytest_cache
rootdir: ##/Testproject_Python-Calculator
plugins: cov-2.6.1
collected 6 items

tst/main.py::CalculatorTest::test_add PASSED                                             [ 16%]
tst/main.py::CalculatorTest::test_divide PASSED                                          [ 33%]
tst/main.py::CalculatorTest::test_factorial PASSED                                       [ 50%]
tst/main.py::CalculatorTest::test_multiply PASSED                                        [ 66%]
tst/main.py::CalculatorTest::test_power PASSED                                           [ 83%]
tst/main.py::CalculatorTest::test_subtract PASSED                                        [100%]

================================== 6 passed in 0.01 seconds ==================================

Testing Frameworks

https://wiki.python.org/moin/PythonTestingToolsTaxonomy

Unit testing framework

import unittest

class TestStringMethods(unittest.TestCase):

    def test_upper(self):
        self.assertEqual('foo'.upper(), 'FOO')

    def test_isupper(self):
        self.assertTrue('FOO'.isupper())
        self.assertFalse('Foo'.isupper())

    def test_split(self):
        s = 'hello world'
        self.assertEqual(s.split(), ['hello', 'world'])
        
        with self.assertRaises(TypeError):
            s.split(2)

if __name__ == '__main__':
    unittest.main()

pytest – helps you write better programms

# content of test_sample.py
def inc(x):
    return x + 1

def test_answer():
    assert inc(3) == 5
$ pytest

nose – is nicer testing for python

def test_numbers_3_4():
    assert multiply(3,4) == 12 
 
def test_strings_a_3():
    assert multiply('a',3) == 'aaa

Python BDD Pattern

class MangoUseCase(TestCase):
  def setUp(self):
    self.user = 'placeholder'

  @mango.given('I am logged-in')
  def test_profile(self):
    self.given.profile = 'profile'
    self.given.photo = 'photo'

    self.given.notifications = 3
    self.given.notifications_unread = 1

    @mango.when('I click profile')
    def when_click_profile():
      print('click')

      @mango.then('I see profile')
      def then_profile():
        self.assertEqual(self.given.profile, 'profile')

      @mango.then('I see my photo')
        def then_photo():
          self.assertEqual(self.given.photo, 'photo')

radsh is not just another BDD tool …THE ROOT FROM RED TO GREEN

from radish import given, when, then

@given("I have the numbers {number1:g} and {number2:g}")
def have_numbers(step, number1, number2):
    step.context.number1 = number1
    step.context.number2 = number2

@when("I sum them")
def sum_numbers(step):
    step.context.result = step.context.number1 + \
        step.context.number2

@then("I expect the result to be {result:g}")
def expect_result(step, result):
    assert step.context.result == result

doctest

"""
The example module supplies one function, factorial().  For example,

>>> factorial(5)
120
"""

def factorial(n):
    """Return the factorial of n, an exact integer >= 0.

    >>> [factorial(n) for n in range(6)]
    [1, 1, 2, 6, 24, 120]
    >>> factorial(30)
    265252859812191058636308480000000
    >>> factorial(-1)
    Traceback (most recent call last):
        ...
    ValueError: n must be >= 0

    Factorials of floats are OK, but the float must be an exact integer:
    >>> factorial(30.1)
    Traceback (most recent call last):
        ...
    ValueError: n must be exact integer
    >>> factorial(30.0)
    265252859812191058636308480000000

    It must also not be ridiculously large:
    >>> factorial(1e100)
    Traceback (most recent call last):
        ...
    OverflowError: n too large
    """

    import math
    if not n >= 0:
        raise ValueError("n must be >= 0")
    if math.floor(n) != n:
        raise ValueError("n must be exact integer")
    if n+1 == n:  # catch a value like 1e300
        raise OverflowError("n too large")
    result = 1
    factor = 2
    while factor <= n:
        result *= factor
        factor += 1
    return result

if __name__ == "__main__":
    import doctest
    doctest.testmod()

Sample Session with Test Frameworks

$ py.test -v
========================================================= test session starts ==========================================================
platform darwin -- Python 3.7.3, pytest-4.3.1, py-1.8.0, pluggy-0.9.0 -- /CLOUD/Development.Anaconda/anaconda3/bin/python
cachedir: .pytest_cache
rootdir: /CLOUD/Development.Python/Repositories.FromGithub/repositories/python-toolbox/Working-with-TDD/app, inifile:
plugins: remotedata-0.3.1, openfiles-0.3.2, doctestplus-0.3.0, arraydiff-0.3
collected 4 items

test_base.py::test_should_pass PASSED                                                                                            [ 25%]
test_base.py::test_should_raise_error PASSED                                                                                     [ 50%]
test_base.py::test_check_if_true_is_true PASSED                                                                                  [ 75%]
test_base.py::test_check_if_inc_works PASSED
$ nosetests -v
test_base.test_should_pass ... ok
test_base.test_should_raise_error ... ok
test_base.test_check_if_true_is_true ... ok
test_base.test_check_if_inc_works ... ok

----------------------------------------------------------------------
Ran 4 tests in 0.001s

OK

Links and additional information

http://pythontesting.net/

https://www.xenonstack.com/blog/test-driven-development-big-data/

https://realpython.com/python-testing/

Flask | Cookbook

Installation

$ pip install flask
$ flask --version
Python 3.7.3
Flask 1.1.1
Werkzeug 0.15.5

Creating a App

Create base python script app.py

from flask import Flask

app = Flask(__name__)

@app.route('/')
def example():
   return '{"name":"Bob"}'

if __name__ == '__main__':
    app.run()

Start Flask

flask run
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
127.0.0.1 - - [01/Aug/2019 12:19:00] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [01/Aug/2019 12:19:00] "GET /favicon.ico HTTP/1.1" 404 -

Jupyter | Cookbook

Cookbook

Customize start dir

$ jupyter notebook --generate-config<br>
Writing default config to: /Users/demo/.jupyter/jupyter_notebook_config.py

Search for the following line in the file

#c.NotebookApp.notebook_dir = ''

Replace by

c.NotebookApp.notebook_dir = '†/the/path/to/home/folder/'

Change password

jupyter lab password

Free online Notebooks

https://notebooks.azure.com/

https://nbviewer.jupyter.org/

https://colab.research.google.com/notebooks/welcome.ipynb

Python | Working with virtual environments

Introduction

  • venv module
  • virtualenv
  • pipenv

pipenv

Introduction

Pipenv — the officially recommended Python packaging tool from Python.org, free (as in freedom).

Read here how to install in details

Installation

For Mac OS and brew package manager

brew install pipenv

Prepare environment

venv module

Installation

Nothing required. Module venv is part of the Python distribution

Prepare environment

python3 -m venv env

virtualenv

Prepare environment

$ virtualenv -p python3 .env/python