공부

[Spark] Exception“org.apache.spark.shuffle.FetchFailedException: Failed to connect”

승가비 2019. 11. 19. 00:45
728x90

You need merged.txt in one partition

[AS-IS]
/yyyy=2019/a.txt
/yyyy=2019/b.txt
/yyyy=2019/c.txt

[TO-BE]
/yyyy=2019/merged.txt

[출처] https://stackoverflow.com/questions/39268624/how-to-fix-org-apache-spark-shuffle-fetchfailedexception-failed-to-connect-in/58918235#58918235

728x90