Sequence and Traverse Part 1 181028181403
Sequence and Traverse Part 1 181028181403
Part 1
learn about the sequence and traverse functions
through the work of
slides by @philip_schwarz
There turns out to be a startling number of operations
that can be defined in the most general possible way in
terms of sequence and/or traverse
Luka Jacobowitz
@LukaJacobowitz
Combines a list of Options into one Option containing a list of all the Some values in the original
list. If the original list contains None even once, the result of the function is None; otherwise the
result is Some with a list of all the values.
if the list is empty then the result is Some empty list Functional Programming in Scala
(by Paul Chiusano and Runar Bjarnason)
@pchiusano @runarorama
assert( sequence(Nil) == Some(List()) )
assert( sequence(List()) == Some(List()) )
if the list contains all Some values then the result is Some list
assert( sequence(List(Some(1))) == Some(List(1)) )
assert( sequence(List(Some(1),Some(2))) == Some(List(1, 2)) )
assert( sequence(List(Some(1),Some(2),Some(3))) == Some(List(1, 2, 3)) )
if the list contains any None value then the result is None
assert( sequence(List(None)) == None )
assert( sequence(List(Some(1),None,Some(3))) == None )
assert( sequence(List(None,None,None)) == None )
Implementing the sequence function
Here’s an explicit recursive version:
def sequence[A](a: List[Option[A]]): Option[List[A]] = a match {
case Nil => Some(Nil)
case h :: t => h flatMap (hh => sequence(t) map (hh :: _))
}
For example, what if we have a whole list of String values that we wish to parse to
Option[Int]? In that case, we can simply sequence the results of the map.
import scala.util.Try
Wanting to sequence the results of a map this way is a common enough occurrence to warrant a
new generic function, traverse
But this is inefficient, since it traverses the list twice, first to convert each String to an Option[Int], and a second pass
to combine these Option[Int] values into an Option[List[Int]].
Better implementations of the traverse function
Here’s an explicit recursive implementation using map2:
Just in case it helps, let’s compare the implementation of traverse with that of sequence
@philip_schwarz
Recall two of the three minimal sets of combinators that can be used to define a monad.
One set includes flatMap and the other includes join (in Scala this is also known as flatten)
@philip_schwarz
flatmap + unit
trait Monad[F[_]] { The flatMap function takes an F[A] and a function from A to F[B] and
def flatMap[A,B](ma: F[A])(f: A => F[B]): F[B] returns an F[B]
def unit[A](a: => A): F[A]
} def flatMap[A,B](ma: F[A])(f: A ⇒ F[B]): F[B]
So flatMapping a function is just mapping the function first and then flattening the result
Now recall that traverse can be defined in terms of map and sequence:
Before we can look at why that is the case, let’s quickly recap what a natural
@philip_schwarz
transformation is.
https://www.slideshare.net/pjschwarz/natural-transformations
@philip_schwarz
Concrete Scala Example: safeHead - natural transformation 𝜏 from List functor to Option functor
val length: String => Int = s => s.length natural transformation 𝜏 from List to Option F[A] is type A lifted into context F
𝜏 f↑F is function f lifted into context F
// a natural transformation
def safeHead[A]: List[A] => Option[A] = { List Option map lifts f into F
case head::_ => Some(head) the square commutes f↑F is map f
case Nil => None safeHead ∘ length↑List = length↑Option ∘ safeHead
} safeHead ∘ (mapList length) = (mapOption length) ∘ safeHead
Naturality
Condition 𝜏 = safeHead
List Option
C1 List C2
𝜏String
List[String] Option[String]
𝜏Int
length↑List List[Int] Option[Int]
List[Int] … … …
List[String] 𝜏Char
the square commutes
List[Char] Option[Char]
length safeHead ∘length↑List
String Int safeHead[String] = 𝜏String 𝜏Int = safeHead[Int]
length↑Option ∘ safeHead
covariant
Category of
Option[String] Option[Int] C1 = C2 = types and functions
length↑Option
https://www.slideshare.net/pjschwarz/natural-transformations
Option @philip_schwarz
Concrete Scala Example: safeHead - natural transformation 𝜏 from List functor to Option functor
trait Functor[F[_]] {
def map[A, B](f: A => B): F[A] => F[B]
}
val mapAndThenTransform: List[String] => Option[Int] = safeHead compose (listF map length)
val transformAndThenMap: List[String] => Option[Int] = (optionF map length) compose safeHead
assert(mapAndThenTransform(List()) == transformAndThenMap(List()))
assert(mapAndThenTransform(List()) == None) https://www.slideshare.net/pjschwarz/natural-transformations
assert(transformAndThenMap(List()) == None)
@philip_schwarz
Having recapped what a natural transformation is,
let’s see why join is a natural transformation.
@philip_schwarz
In Category Theory a Monad is a functor equipped with a pair of natural transformations satisfying the laws of associativity and identity
What does this mean? If we restrict ourselves to the category of Scala types (with Scala types as the objects
and functions as the arrows), we can state this in Scala terms.
trait Functor[F[_]] {
def map[A,B](fa: F[A])(f: A => B): F[B]
}
type Id[A] = A
@philip_schwarz
safeHead: List[A] ⇒ Option[A]
unit: A ⇒ List[A]
join: List[List[A]] ⇒ List[A]
sequence: List[Option[A]] ⇒ Option[List[A]]
It seems to me that sequence is a natural transformation, like safeHead, but in some higher-order sense. Is that right?
• natural transformation safeHead maps the List Functor to the Option Functor
• I can either first apply safeHead to a list and then map the length function over the result
• Or I can first map the length function over the list and then apply safeHead to the result
• The overall result is the same
• In the first case, map is used to lift the length function into the List Functor
• In the second case, map is used to lift the length function into the Option Functor
Is it correct to consider sequence to be a natural transformation? I ask because there seems to be something higher-order about sequence
compared to safeHead
• natural transformation sequence maps a List Functor of an Option Functor to an Option Functor of a List Functor
• I can either first apply sequence to a list of options and then map a function that maps the length function
• Or I can first map over the list a function that maps the length function, and then sequence the result
• The overall result is the same
• In the first case, we first use map to lift the length function into the List Functor and then again to lift the resulting function into the Option
Functor,
• In the second case, we first use map to lift the length function into the Option Functor and then again to lift the resulting function into the List
Functor
It seems that for a natural transformation that rearranges N layers of Functors we call map on each of those layers before we apply a function.
So yes, sequence, like safeHead, unit and join, is a natural transformation.
They are all polymorphic functions from one functor to another.
@philip_schwarz
Let’s illustrate this further in the next slide using diagrams and some code.
Option[String] Option[Int]
length↑O val expected = Some(2)
// first map and then transform
length↑O∘ safeHeadList assert(safeHead(List("ab","abc","a").map(_.length)) == expected)
// first transform and then map
safeHeadList safeHeadOption assert((safeHead(List("ab","abc","a")).map(_.length)) == expected)
join join
val expected = Some(List(2,3,1))
join ∘(length↑L)↑L // first map and then transform
assert(sequence(List(Some("ab"),Some("abc"),Some("a")).map.(_.map(_.length))) == expected)
(length↑L)↑L // first transform and then map
List[List[String]] List[List[Int]] assert(sequence(List(Some("ab"),Some("abc"),Some("a"))).map(_.map(_.length)) == expected)
Combines a list of Options into one Option containing a list of all the Some values in the original list. If
the original list contains None even once, the result of the function is None ; otherwise the
result is Some with a list of all the values.
Combines a list of Eithers into one Either containing a list of all the Right values in the original list.
If the original list contains Left even once, the result of the function is the first Left; otherwise the
result is Right with a list of all the values.
The sequence function for Either
def sequence[E,A](a: List[Either[E,A]]): Either[E,List[A]]
Combines a list of Eithers into one Either containing a list of all the Right values in the original
list. If the original list contains Left even once, the result of the function is the first Left; otherwise
the result is Right with a list of all the values.
if the list contains all Right values then the result is Right of a list
assert( sequence(List(Right(1))) == Right(List(1)) )
assert( sequence(List(Right(1),Right(2))) == Right(List(1, 2)) )
assert( sequence(List(Right(1),Right(2),Right(3))) == Right(List(1, 2, 3)) )
if the list contains any Left value then the result is the first Left value
assert( sequence(List(Left(-1))) == Left(-1) )
assert( sequence(List(Right(1),Left(-2),Right(3))) == Left(-2) )
assert( sequence(List(Left(0),Left(-1),Left(-2))) == Left(0) )
From implementation of sequence for Option to implementation of sequence for Either
explicit recursive
implementation
def traverse[A,B,E](a: List[A])(f: A => Either[E, B]): Either[E,List[B]] = a match {
case Nil => Right(Nil)
case h::t => map2(f(h),traverse(t)(f))(_ :: _)
}
Implementation using
foldRight and map2
For example, what if we have a whole list of String values that we wish to parse to Either[Throwable,Int]? In that case, we can simply sequence the
results of the map.
import scala.util.Try
@philip_schwarz
def sequence[A](a: List[Option[A]]): Option[List[A]] =
a.foldRight[Option[List[A]]](Some(Nil))((h,t) => map2(h,t)(_ :: _))
E.g. in the above example, the Option specific items that sequence, traverse and map2 depend on are Option’s
Some constructor, Option’s map function, and Option’s flatMap function. In the case of Either, the dependencies
are on Either’s Right constructor, Either’s map function and Either’s flatMap function.
Option and Either are monads and every monad has a unit function, a map function, and a flatMap function.
Since Some and Right are Option and Either’s unit functions and since map2 can be implemented using map and
flatMap, it follows that sequence and traverse can be implemented for every monad.
See the next slide for a reminder that every monad has unit, map and flatMap functions.
flatmap + unit
trait Monad[F[_]] {
def flatMap[A,B](ma: F[A])(f: A => F[B]): F[B] = join(map(ma)(f)) The three different ways of
}
def compose[A,B,C](f: A => F[B], g: B => F[C]): A => F[C] = a => flatMap(f(a))(g)
defining a monad, and how
a monad always has the
Kleisli composition + unit following three functions:
trait Monad[F[_]] { unit, map, flatMap
def compose[A,B,C](f: A => F[B], g: B => F[C]): A => F[C]
def unit[A](a: => A): F[A]
def flatMap[A,B](ma: F[A])(f: A => F[B]): F[B] = compose((_:Unit) => ma, f)(())
def map[A,B](m: F[A])(f: A => B): F[B] = flatMap(m)(a => unit(f(a)))
}
Generalising the signatures of map2, sequence and traverse so they work on a monad F
}
A simple example of using the traversable function of the Option monad
val optionM = new Monad[Option] {