티스토리 뷰

공부

[Spark] MySQL jdbc write `rewriteBatchedStatements=true`

승가비 2020. 10. 28. 05:53
728x90

https://stackoverflow.com/questions/36912442/low-jdbc-write-speed-from-spark-to-mysql

 

Low JDBC write speed from Spark to MySQL

I need write about 1 million rows from Spark a DataFrame to MySQL but the insert is too slow. How can I improve it? Code below: df = sqlContext.createDataFrame(rdd, schema) df.write.jdbc(url='x...

stackoverflow.com

 

728x90
저작자표시 비영리

'공부' 카테고리의 다른 글

[Spark] JOIN spark.sql.shuffle.partitions  (0) 2020.10.28
[Spark] jdbc 사용시 주의할 사항  (0) 2020.10.28
[JS] screenshot of div  (0) 2020.10.28
[Java] random shuffle list  (0) 2020.10.28
[JS] date deep copy  (0) 2020.10.28
댓글