티스토리 뷰

공부

[Spark] What's the difference between --archives, --files, py-files in pyspark job arguments

승가비 2021. 2. 25. 12:43
728x90

stackoverflow.com/questions/38066318/whats-the-difference-between-archives-files-py-files-in-pyspark-job-argum

 

What's the difference between --archives, --files, py-files in pyspark job arguments

--archives, --files, --py-files and sc.addFile and sc.addPyFile are quite confusing, can someone explain these clearly?

stackoverflow.com

 

728x90
저작자표시 비영리

'공부' 카테고리의 다른 글

[Sh] date -d '+1 hour' '+%Y %m%d%H' (yyyyMMddHH=`date +%Y%m%d%H`) `date +%Y%m%d -d '1 days ago'`  (0) 2021.03.01
[Python] Extracting an attribute value with beautifulsoup  (0) 2021.02.26
[HDFS] update file (get & put))  (0) 2021.02.25
[Java] StringBuffer, StringBuilder 가 String 보다 성능이 좋은 이유와 원리  (0) 2021.02.10
[Hive] ORC table  (0) 2021.02.10
댓글