티스토리 뷰

공부

[Spark] `repartition` vs `coalesce`

승가비 2022. 11. 7. 20:25
728x90

repartition: swap

coalesce: repartition optimized

 

https://sparkbyexamples.com/spark/spark-repartition-vs-coalesce/

 

Spark Repartition() vs Coalesce()

Spark repartition() vs coalesce() - repartition() is used to increase or decrease the RDD, DataFrame, Dataset partitions whereas the coalesce() is used to only decrease the number of partitions in an efficient way. In this article, you will learn what is S

sparkbyexamples.com

 

728x90
저작자표시 비영리 (새창열림)

'공부' 카테고리의 다른 글

[Spark] withColumnrenamed  (0) 2022.11.07
[Python][smtplib] Gmail  (0) 2022.11.07
[Gradle] Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain  (0) 2022.11.07
[Spark] dataframe to table  (0) 2022.11.07
[HBase] tuning  (0) 2022.11.07
댓글