work/1-intro.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "**Goals:** This notebook is meant to serve as an introduction to [Scala](http://scala-lang.org/) and [Apache Spark](https://spark.apache.org/).\n",
    "\n",
    "## Scala \n",
    "This is a basic introduction to assignments, IO, and functions in Scala."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "// A simple variable assignment\n",
    "val x = 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "// We can print values to standard output\n",
    "println(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "// A simple function declaration to add two integers\n",
    "def add(a: Int, b: Int) = {\n",
    "    a + b\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "add(1,5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Spark Intro\n",
    "Toree creates several objects used to interact with Spark. The most common is the [SparkContext](https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.SparkContext) object which is bound to the variable `sc`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "// We create a Spark RDD of the numbers 1 to 100\n",
    "val numbersRDD = sc.parallelize(1 to 100)\n",
    "numbersRDD.collect()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "// We map the numbers to be a tuple of the number and a boolean stating if it is divisble by two\n",
    "val mappedNumbersRDD = numbersRDD.map((num: Int) => {\n",
    "    (num, num % 2 == 0)\n",
    "})\n",
    "mappedNumbersRDD.collect()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "// We filter the RDD to get only the even numbers\n",
    "val evenNumbersRDD = mappedNumbersRDD.filter((tuple: (Int, Boolean)) => {\n",
    "    tuple._2\n",
    "})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "// Collect the numbers for output and them out with a message\n",
    "evenNumbersRDD.collect().foreach( x => println(s\"${x._1} is an even number\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Apache Toree - Scala",
   "language": "scala",
   "name": "apache_toree_scala"
  },
  "language_info": {
   "name": "scala"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}