Apache Spark is a lightning-fast cluster computing designed for fast computation. It was built on top of Hadoop MapReduce and it extends the MapReduce model to efficiently use more types of computations which includes Interactive Queries and Stream Processing.

This is an extract from this brief tutorial that explains the basics of Spark Core programming.

Environment / Requirements

• Spark
• Java
• Scala

Installation on Mac OS X

Check or install java

$ java -version
java version "12.0.1" 2019-04-16
Java(TM) SE Runtime Environment (build 12.0.1+12)
Java HotSpot(TM) 64-Bit Server VM (build 12.0.1+12, mixed mode, sharing)

Check or install Scala

$ brew install scala

$ scala -version
Scala code runner version 2.13.0 -- Copyright 2002-2019, LAMP/EPFL and Lightbend, Inc.

Check or install Apache Spark

Setup environment in .bashrc

export PATH="$PATH:$SPARK_HOME/bin"

Installation on Ubuntu

Prepate Upuntu

apt update
apt upgrade

 apt-get install openjdk-8-jdk
 java -version

Links and Resources

• Apache Spark in Python: Beginner’s Guide

TL;DR

Combine two amazing open source tools: Jekyll and Bootstrap. The final template is here.

Start Point

While i want to learn about and work with bootstrap, i decided to build a Jekyll Template, so that i can build a dynamic website.

Asking Google for first inspiration leads me to this wonderful Blog entry:

Choose a Bootstrap Template

Quite nice. So I decided to use one of the free templates from Start Bootstrap: Modern Business

When i downloaded the template from Github and examine the content, i find out, that for each component (Pricing, Service, Contact), there is a corresponding HTML-file with all the content and all the formatting code:

• about.html
• blog-home-1.html
• blog-home-2.html
• blog-post.html
• contact.html
• faq.html
• full-width.html
• index.html
• portfolio-1-col.html
• portfolio-2-col.html
• portfolio-3-col.html
• portfolio-4-col.html
• portfolio-item.html
• pricing.html
• services.html
• sidebar.html

The Plan

My plan was to separate the presentation layer (what you will see) from the business layer (what creates the content for the presentation layer).

To achieve this with Jekyll, i convert the Bootstrap pages to Jekyll include pages. The final result should look like this:

---
layout: page
title: Services
---
<div class="container">
    <h1 class="mt-4 mb-3">{{ page.title }}</h1>
    
</div>

<h2>Services: {{ site.services.title }}</h2>

<!-- Image Header -->
<img class="img-fluid rounded mb-4" src="{{ images }}/header.jpg" alt="">

<div class="row">
    
        <div class="col-lg-4 mb-4">
            <div class="card h-100">
                <h4 class="card-header">{{ item.title }}</h4>
                <div class="card-body">
                    <p class="card-text">{{ item.text | markdownify }}</p>
                </div>
                <div class="card-footer">
                    <a href="#" class="btn btn-primary">Learn More</a>
                </div>
            </div>
        </div>
    
</div>

Next step was to convert every Bootstrap Template Page to a Jekyll Include File

About Page

FAQ Page

Portfolio Page with 1 Column

Portfolio Page with 2 Column

Services Page

Pricing Page

The main challenge in separating the presentation from the business layer was: where to place the data to be displayed?

Depending on the type of the component, i choose three different solutions:

1. Place the data in the corresponding include file of the component
2. Place the date in the page, which calls the corresponding include file of the component
3. Place the data in a Jekyll collection file

Data in corresponding include file of the component

I used this approach for components, which are used only once on the website and have a mostly static content, e.g. the FAQ Page

The component page

The frontend page

Date in the page, which calls the corresponding include file of the component

I used this approach for components, which are used more than once on the website, e.g. a Blog Post

The component page

The frontend page

Data in a Jekyll collection file

I used this approach for components, which are used only once on the website, but needs more configuration information, e.g. the Services- or Portfolio Page.

This step needs an additional configuration task: create the Jekyll Collections.

Jekyll collections are a great way to group related content like members of a team or talks at a conference.

To use a Collection you first need to define it in your _config.yml.

#
collections_dir: collections # folder, where collections files are stored
collections:
  services:
    title: "Services"
    output: true # store output files for each item under the collections folder

Then, you have to create the collection files, for each item in your collection one file:

These files look like this:

---
img: 1.jpg
title: Development
subtitle: 
footer: 
text: Lorem ipsum dolor sit amet, consectetur adipisicing elit. Possimus aut mollitia eum ipsum fugiat odio officiis odit.
---

And the data of this files can be accessed in the Jekyll include file with this code fragment:

• all items of the collection: site.services,

      <div class="col-lg-4 mb-4">
          <div class="card h-100">
              <h4 class="card-header">{{ item.title }}</h4>
              <div class="card-body">
                  <p class="card-text">{{ item.text | markdownify }}</p>
              </div>
              <div class="card-footer">
                  <a href="#" class="btn btn-primary">Learn More</a>
              </div>
          </div>
      </div>
  
  
  
  
  The final result