When a resilient distributed dataset (RDD) is created from a text file or collection (or from another RDD), do we need to call “cache” or “persist” explicitly to store the RDD data into memory? Or is the RDD data stored in a distributed way in the memory by default? val textFile = sc.textFile(“/user/emp.txt”) As per … Read more
One way that has been suggested to deal with double definitions of overloaded methods is to replace overloading with pattern matching: object Bar { def foo(xs: Any*) = xs foreach { case _:String => println(“str”) case _:Int => println(“int”) case _ => throw new UglyRuntimeException() } } This approach requires that we surrender static type … Read more
True… it has been discussed quite a lot. However, there is a lot of ambiguity and some of the answers provided … including duplicating JAR references in the jars/executor/driver configuration or options. The ambiguous and/or omitted details The following ambiguity, unclear, and/or omitted details should be clarified for each option: How ClassPath is affected Driver … Read more
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers. Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done … Read more
When should I use reduceLeft, reduceRight, foldLeft, foldRight, scanLeft or scanRight? I want an intuition/overview of their differences – possibly with some simple examples. 3 Answers 3
Given a variable with type Graphics, how do I cast it to Graphics2D in Scala? 2 Answers 2
I prefer Python over Scala. But, as Spark is natively written in Scala, I was expecting my code to run faster in the Scala than the Python version for obvious reasons. With that assumption, I thought to learn & write the Scala version of some very common preprocessing code for some 1 GB of data. Data … Read more
In scala.collection, there are two very similar objects JavaConversions and JavaConverters. What is the difference between these two objects? Why do they both exist? When do I want to use one vs. the other? 4 Answers 4
I can see in the API docs for Predef that they’re subclasses of a generic function type (From) => To, but that’s all it says. Um, what? Maybe there’s documentation somewhere, but search engines don’t handle “names” like “<:<” very well, so I haven’t been able to find it. Follow-up question: when should I use … Read more
It seems that Vector was late to the Scala collections party, and all the influential blog posts had already left. In Java ArrayList is the default collection – I might use LinkedList but only when I’ve thought through an algorithm and care enough to optimise. In Scala should I be using Vector as my default … Read more
