티스토리 뷰

공부

[spark] executor-cores = 5 (recommended)

승가비 2023. 3. 6. 00:53
728x90

https://aws.amazon.com/ko/blogs/big-data/best-practices-for-successfully-managing-memory-for-apache-spark-applications-on-amazon-emr/

 

Best practices for successfully managing memory for Apache Spark applications on Amazon EMR | Amazon Web Services

May 2022: Post was reviewed for accuracy. Since this post has been published, Amazon EMR has introduced several new features that make it easier to fully utilize your cluster resources by default. The default Apache Spark configurations were updated in EMR

aws.amazon.com

 

728x90
저작자표시 비영리

'공부' 카테고리의 다른 글

[flutter] minSdkVersion Declared in library  (0) 2023.03.06
[terraform] terraform apply -auto-approve  (0) 2023.03.06
[spark] DFSClient: DFSOutputStream ResponseProcessor exception for block BP-1565110817-10.161.67.203-1521280878051:blk_1440726264_367066900java.io.EOFException: Premature EOF: no length prefix available at org.apache.hadoop.hdfs.protocolPB.PBHelper..  (0) 2023.03.06
[spark] drop view (createTempView)  (0) 2023.03.06
[spark] AQE (Adaptive Query Execution)  (0) 2023.03.01
댓글