Scala Training

• Day 1:
• Introduction to Scala
• Scala vs Java
• Installation and setup Scala
• Datatypes, variables
• Conditions and Loops statements
• Functions
High order functions
Anonymous functions
Partially applied functions
Function with variable arguments
Function call-by-name

• Access Modifiers
• Day 2:
• Basic object orient programming
Classes
Immutable and Mutable Fields
Methods
Default and Named Arguments
Objects

• Case Objects and Classes

 Companion Objects
 Case Classes and Case Objects
• Collections
List
Set
Maps
Tuples
Options & iterators

• Extractors
• Pattern matching
• Day 3:
• Exception handling
• File handling
• Introduction to Spark
Spark architecture
Introduction to Big Data and Hadoop
Spark Framework
• RDD Vs DSM
• RDD’s Features
• RDD Transformation and operations
• Day 4:
• Hands on program for RDD
Data Loading and Saving Through RDDs
Key-Value Pair RDDs and Other Pair RDDs
RDD Persistence
RDD Partitions
RDD Lineage
• Data frames and Spark SQL
• Spark SQL architecture
• Data Frames & Datasets
• JSON and Parquet File Formats(loading data into Hive)
• Loading Data through Different Sources
• Day 5:
• Uses cases/Sample project
• Code and development
Business Problem
Decline in the no. of transactions on a Bank’s credit/
debit cards.

Existing Strategy: One to Many promotion, One size fits all

New Strategy: One to One promotion ,“ Right place, Right time, Relevant Promotion”
 Non-Relevant Promotion

Customer is agitated since he is getting non-relevant

promotions continuously on his mobile
Student(Customer) enters a shopping
mall to have lunch in a restaurant
Customer purchase history
and loyalty preference
indicate his inclination for a
restaurant within the same
mall

He receives
an SMS
Get 2% off on your next purchase in the
Jewelry store within 20 minutes.
Wrong Place

Customer is driving a Car

and is stuck in traffic

Customer is agitated since he is getting

promotions in the wrong place
continuously on his mobile
Receives a call from bank
regarding a promotion offer
on their credit card

Get a pre approved loan

based on the historical
performance
 Wrong time

Customer is attending
Nature’s call (7 AM)

Customer is angry since this is the third time

that he has received a call at a wrong time.

Get 5% off if you purchase a Home

Theatre system within the next 20
minutes
Receives a call from the Bank
 Why BIG DATA
Big data capabilities allowed the bank to process and analyze nearly 0.05 TB of structured
and unstructured data within a span of few seconds to come up with insightful inferences
and deliver the relevant promotion at the right place and at the right time.
 Data Attributes
Purchase History Communication Channel preference

GPS location

Social Media Posts and Tweets

Loyalty preferences
 What it means in terms of BIG DATA…
GPS location Feeds from target segments: Mon-Fri ( 7 PM -11PM), Sat-Sun (10 AM -11PM)

0.01 Terabyte of data to be processed within few seconds during weekends, 0.005 TB of data in
off-peak hours

Social Media posts and Tweets

Purchase History
0.02 TB of unstructured social media data to be
analyzed within few seconds.

0.02 TB of purchase history data of the target segments

to be analyzed within few seconds.
 Right Place, Right Time, Relevant Promotion

Customer enters the

shoe store and ends
up buying a pair of
shoes.
GPS location of customer
tracked: he is in a Mall
Get 50% off on min. billing amount of
Rs 5000 , for the next 20 minutes.
Valid on CELOS shoe store only

Customer purchase history and loyalty

preference indicate his fetish for shoes,
social media posts indicate the quality
of products in “CELOS shoe store” is
extremely good
Customer receives an SMS when he
reaches the “CELOS shoe store
Value to Company & Customer
Spark
Supporting APIs
Why Spark
Spark component
 Scalable Language
• Scala is a modern muliti-paradigm programming language
• Statically Typed
• Runs on JVM, full inter-op with JAVA
• Object oriented
• Functional
• Dynamic features
 Dynamic Vs Static
Dynamic Static
• Concise • Better IDE support
• Scriptable • Fewer Tests
• REPL(Read Evaluate print Loop) • Documentation
• High order functions • Opensource libs
• Extend existing classes • Performance
• Duck Typing • JVM tools
• True multithreading
 Scala Vs Java
• Similarities:
• Both are using JVM and generate byte using scalac like Javac
• Inside Scala we can use Java code vice versa.
• Major IDEs support Scala
• Both are supporting OOPs.
• Difference:
• Scala support functional programming
• Lazy evaluator
• In Scala everything is objects.
• new features like type inference, a powerful type system, pattern matching,
traits, and higher order functions.
Continued..
• Object Oriented • Lazy Evaluation
• Functional programming • Readability
• Multi core CPU • Operator Overloading
• architecture • Learning curve
• Compiler • Compile Time
• JVM • Programming error
• IDE support • Finding Error
• REPL application • Backward Compatibility
• Succinct and Concise code • Dynamic Behaviour
 Scala REPL
• Scala> 2017+1
• Scala> 2017.()+1
• Scala> val a = 10
• Scala> a=20 ?
• Scala>: imports
• Scala>: helps
Scala basic data types
Name Description Size Min Max

Byte Signed integer 1 byte –127 128

Short Signed integer 2 bytes –32768 32767

Int Signed integer 4 bytes –231 231–1

Long Signed integer 8 bytes –263 263–1

Float Signed floating point 4 bytes n/a n/a

Double Signed floating point 8 bytes n/a n/a

Boolean Two possible values

Char 16-bit unicode

Literals
scala> val nov = 0777
• nov: Int = 511
scala> val tower = 35L
• tower: Long = 35
scala> val biggerStill = 123E45
biggerStill: Double =
1.23E47
scala> val a = 'A’
a: Char = A
String data types
• Scala> val str = “Hello, world”
• Scala> s “This is one of string feature $str”
• scala> val height = 1.9d
• height: Double = 1.9
• scala> val name = "James"
• name: String = James
• scala> println(f"$name%s is $height%2.2f meters tall")
• James is 1.90 meters tall
Xml Literals
• val book = <Employee>
<title>Software engineer</title>
<name> James</name>
</Employee>
Operator and methods

• scala> val sumMore = (1).+(2) Logical Operator:

• scala> val longSum = 1 + 2L scala> val toBe = true
• scala> val s = "Hello, world!" • toBe: Boolean = true
• scala> s indexOf 'o’ • scala> val question = toBe ||
• scala> s toLowerCase !toBe
• question: Boolean = true
• scala> val paradox = toBe &&
!toBe
• paradox: Boolean = false
Bitwise operators
• scala> 1 & 2
• res24: Int = 0
• scala> 1 | 2
• res25: Int = 3
• scala> 1 ˆ 3
• res26: Int = 2
• scala> ~1
• res27: Int = 2
If then else statement
object Demo {
def main(args: Array[String]) {
var x = 30;

if( x == 10 ){
println("Value of X is 10");
} else if( x == 20 ){
println("Value of X is 20");
} else if( x == 30 ){
println("Value of X is 30");
} else{
println("This is else statement");
}
}
While loop:

object Demo {
def main(args: Array[String]) {
var i=0
while (i<10)
{
println("i value is "+i)
i+=1
}
}
}
Do while loop:

object Helloworld {

def main(args: Array[String]): Unit = {

var line = ""
do {
line = readLine()
println("Read: " + line)
} while (line != "Stop!")
}
}
For loop
for (‘(’ Enumerators ‘)’ | ‘{’ Enumerators ‘}’) {nl} [‘yield’] Expr
object Demo {
def main(args: Array[String]) {
for (i <- 1 to 10){
println("i value is " + i)
}
}
}
object Demo {
def main(args: Array[String]) {
val files = new java.io.File("C:\\scala_training").listFiles
for(file <- files) {
val filename = file.getName
if(filename.endsWith(".scala")) println(file)
}
}
}
 object Demo {
def main(args: Array[String])
{
for (i <- 1 to 10; j <- 1 to 10){
println("i value is " + i +" J value is" + j)
} }
}

object Demo {
def main(args: Array[String])
{
val result = for (i <- 1 to 10; j <- 1 to 10) yield i+j
println(result)
}
}
Functions:
• def functionName ([list of parameters]) : [return type] = {
• function body
• return [expr]
• }
• scala> def myFirstfun() = { “This is a sample function" }
• myFirstMethod: ()java.lang.String
• scala> myFirstfun()
• res1: java.lang.String = This is a sample function
• ---------------------------------------------------------------------------------------
• scala> def myFirstfun(){ “This is a sample function" }
• myFirstfun : ()Unit
• scala> myFirstfun()
 Function with arguments
object Demo {
def main(args: Array[String])
{
println("before function")
myfun(30,"Sara")
def myfun(Age:Int, Name:String): Unit =
{
println("My name is :" +Name)
println("Age is "+Age)
}
}
}
Named parameters
object Demo {
def main(args: Array[String])
{
println("before function")
myfun(30)
myfun(Age = 10, Name = "xyz")
def myfun(Age:Int, Name:String = "unknown"): Unit =
{
println("My name is :" +Name)
println("Age is "+Age)
}
}
}
Functions with variable arguments
object Demo {
def main(args: Array[String]) {
printStrings("Java", "Scala", "Python");
}

def printStrings( args:String* ) = {

var i : Int = 0;

for( arg <- args ){

println("Arg value is = " + arg );
}
}
}
Function Call by values
import java.sql.Date

object Demo {
def main(args: Array[String]) {
delayed(time());
}

def time() = {
println("Getting time in nano seconds")
System.currentTimeMillis()
}
def delayed( t: => Long ) = {
println("In delayed method")
println("Param: " + new Date(t) )
}
}
Higher-order Function
object Demo {
def main(args: Array[String]) {
println( apply( layout, sumInts(1,3)) )
println(sumInts(1,3))
}
def sumInts(a: Int, b: Int) = sum(id, a, b)
def apply(f: Int => String, v: Int) = f(v)
def id(x: Int): Int = x
def layout[A](x: A) = "[" + x.toString() + "]"
def sum(f: Int => Int, a: Int, b: Int): Int =
if (a > b) 0
else f(a) + sum(f, a + 1, b)
}
Anonymous function
• scala> increase = (x: Int) => {
• println("We")
• println("are")
• println("here!")
• x+1
•}
• increase: (Int) => Int =
<function>
Partially called functions
import java.util.Date

object Demo {
def main(args: Array[String]) {
val date = new Date
val logWithDateBound = log(date, _ : String)

logWithDateBound("message1" )
Thread.sleep(1000)

logWithDateBound("message2" )
Thread.sleep(1000)

logWithDateBound("message3" )
}

def log(date: Date, message: String) = {

println(date + "----" + message)
}
Closure
import java.util.Date

object Demo {
def main(args: Array[String]) {
var closure_val = 10
def addclosure(x:Int): Unit =
{
println(x+closure_val)
}
addclosure(1)
closure_val = 100
addclosure(1)
}
}
Class
Classes in Scala are static templates that can be instantiated into many
objects at runtime
A Class can contain information about:
• Fields
• Constructors
• Methods
• Superclasses(inheritance)
• Interfaces implemented by the class, etc.
Example
object Demo {
def main(args: Array[String]) {
class addition{
var x:Int=10
def add(a:Int,b:Int)
{
x = x+a
println("Value of x after modification :"+ x)
}
}
val p=new addition()
p.add(5,10);
}
}

class Employee {

def display_emp(empno:Int, ename:String, salary: Double): Unit =

{
println("Empno :"+empno +"\n employee name :"+ename + "\nSalary"+salary)
}

}
class Point(xc: Int, yc: Int) {
var x: Int = xc
var y: Int = yc

def move(dx: Int, dy: Int) {

x = x + dx
y = y + dy
println ("Point x location : " + x);
println ("Point y location : " + y);
}
}
object demo
{
def main(args: Array[String]): Unit = {
val p = new Point(10,20)
p.move(10,20)
}