티스토리 뷰

공부

[spark] `spark.yarn.stagingDir`

승가비 2022. 12. 6. 13:44
728x90

https://stackoverflow.com/questions/42723604/how-to-set-spark-job-staging-location

 

How to set Spark job staging location

My spark job is failing because the user doesn't have access to directory where spark is trying to write staging or temp dataset. 2017-03-10 10:25:47,0928 ERROR JniCommon fs/client/fileclient...

stackoverflow.com

https://doc.hcs.huawei.com/en-us/usermanual/mrs/mrs_03_0298.html

 

https://doc.hcs.huawei.com/en-us/usermanual/mrs/mrs_03_0298.html

On this page

doc.hcs.huawei.com

 

728x90
저작자표시 비영리 (새창열림)

'공부' 카테고리의 다른 글

[SQL] dynamic like  (0) 2022.12.07
[bitbucket] pipelines `atlassian/aws-s3-deploy:0.3.8` force add sub path  (0) 2022.12.07
[python] jinja2 + f-string  (0) 2022.12.06
[zeppelin] org.apache.spark.sql.execution.datasources.FileDownloadException: Failed to download file path: s3://bucket/prefix/partition.snappy.parquet, range: 0-16165503, partition values: [empty row], isDataPresent: false, eTag: 53ea26b5ecc9a194efe5163..  (0) 2022.12.06
[python] venv deactivate  (0) 2022.12.03
댓글