[zeppelin] spark.yarn.stagingDir (x) -> spark.hadoop.hive.exec.stagingdir (o)
spark.hadoop.hive.exec.stagingdir: s3://temp/.hive-staging https://vanducng.dev/2020/12/05/Compact-multiple-small-files-on-HDFS/ Compact multiple small files on HDFS Hadoop can handle with very big file size, but will encounter performance issue with too many files with small size. The reason is explained in detailed from here. In short, every single on a data n vanducng.dev
공부
2023. 1. 21. 03:18
728x90
728x90