공부
[Spark] Total size of serialized results of 16 tasks bigger than spark.driver.maxResultSize
승가비
2020. 11. 7. 14:31
728x90
--conf spark.driver.maxResultSize=4g
Total size of serialized results of 16 tasks (1048.5 MB) is bigger than spark.driver.maxResultSize (1024.0 MB)
I get the following error when I add --conf spark.driver.maxResultSize=2050 to my spark-submit command. 17/12/27 18:33:19 ERROR TransportResponseHandler: Still have 1 requests outstanding when
stackoverflow.com
728x90