(Why) do we need to call cache or persist on a RDD
When a resilient distributed dataset (RDD) is created from a text file or collection (or from another RDD), do we need to call … Read more
rdd
Spark performance for Scala vs Python
I prefer Python over Scala. But, as Spark is natively written in Scala, I was expecting my code to run faster in the … Read more
What is the difference between cache and persist?
In terms of RDD persistence, what are the differences between cache() and persist() in spark ? 6 Answers 6
Difference between DataFrame, Dataset, and RDD in Spark
I’m just wondering what is the difference between an RDD and DataFrame (Spark 2.0.0 DataFrame is a mere type alias for Dataset[Row]) in … Read more
Spark – repartition() vs coalesce()
According to Learning Spark Keep in mind that repartitioning your data is a fairly expensive operation. Spark also has an optimized version of … Read more