{"id":4921,"date":"2019-06-28T10:37:40","date_gmt":"2019-06-28T08:37:40","guid":{"rendered":"http:\/\/blog.via-internet.de\/?p=4921"},"modified":"2019-06-28T10:37:40","modified_gmt":"2019-06-28T08:37:40","slug":"spark-getting-started","status":"publish","type":"post","link":"https:\/\/via-internet.de\/blog\/2019\/06\/28\/spark-getting-started\/","title":{"rendered":"Apache Spark | Getting started"},"content":{"rendered":"\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>Apache Spark is a lightning-fast cluster computing designed for fast computation. It was built on top of Hadoop MapReduce and it extends the MapReduce model to efficiently use more types of computations which includes Interactive Queries and Stream Processing. <\/p><\/blockquote>\n\n\n\n<p class=\"wp-block-paragraph\">This is an extract from <a href=\"https:\/\/www.tutorialspoint.com\/apache_spark\/\">this<\/a> brief tutorial that explains the basics of Spark Core programming.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Environment \/ Requirements<\/h2>\n\n\n\n<ul class=\"wp-block-list\"><li><a href=\"https:\/\/spark.apache.org\/\">Spark<\/a><\/li><li>Java<\/li><li><a href=\"https:\/\/www.scala-lang.org\/\">Scala<\/a><\/li><\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Installation on Mac OS X<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Check or install java<\/h3>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">$ java -version\njava version \"12.0.1\" 2019-04-16\nJava(TM) SE Runtime Environment (build 12.0.1+12)\nJava HotSpot(TM) 64-Bit Server VM (build 12.0.1+12, mixed mode, sharing)<\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Check or install Scala<\/h3>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">$ brew install scala<\/pre>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">$ scala -version\nScala code runner version 2.13.0 -- Copyright 2002-2019, LAMP\/EPFL and Lightbend, Inc.<\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">Check or install Apache Spark<\/h4>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">Setup environment in .bashrc<\/h4>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">export PATH=\"$PATH:$SPARK_HOME\/bin\"<\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Installation on Ubuntu<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Prepate Upuntu<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"shell\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">apt update\napt upgrade\n<\/pre>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"shell\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"> apt-get install openjdk-8-jdk\n java -version<\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Links and Resources<\/h2>\n\n\n\n<ul class=\"wp-block-list\"><li><a href=\"https:\/\/www.datacamp.com\/community\/tutorials\/apache-spark-python\">Apache Spark in Python: Beginner&#8217;s Guide<\/a><\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Apache Spark is a lightning-fast cluster computing designed for fast computation. It was built on top of Hadoop MapReduce and it extends the MapReduce model to efficiently use more types of computations which includes Interactive Queries and Stream Processing. This is an extract from this brief tutorial that explains the basics of Spark Core programming. Environment \/ Requirements Spark Java Scala Installation on Mac OS X Check or install java Check or install Scala Check or install Apache Spark Setup environment in .bashrc Installation on Ubuntu Prepate Upuntu Links and Resources Apache Spark in Python: Beginner&#8217;s Guide<\/p>\n","protected":false},"author":1,"featured_media":4922,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_crdt_document":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[4,5],"tags":[],"class_list":["post-4921","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-apache","category-apache-spark"],"jetpack_featured_media_url":"https:\/\/via-internet.de\/blog\/wp-content\/uploads\/2019\/06\/logo-apache-spark-mini-logo.jpg","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/posts\/4921","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/comments?post=4921"}],"version-history":[{"count":0,"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/posts\/4921\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/media\/4922"}],"wp:attachment":[{"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/media?parent=4921"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/categories?post=4921"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/tags?post=4921"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}