티스토리 뷰

공부

[Spark] to_json (need to spark 2.1)

승가비 2020. 10. 3. 20:08
728x90
import org.apache.spark.sql.functions.to_json
df.select(to_json(struct($"c1", $"c2", $"c3")))

https://stackoverflow.com/questions/36157810/spark-row-to-json

 

Spark Row to JSON

I would like to create a JSON from a Spark v.1.6 (using scala) dataframe. I know that there is the simple solution of doing df.toJSON. However, my problem looks a bit different. Consider for insta...

stackoverflow.com

https://spark.apache.org/docs/2.1.0/api/java/org/apache/spark/SparkFiles.html#getRootDirectory()

 

SparkFiles (Spark 2.1.0 JavaDoc)

static String get(String filename) Get the absolute path of a file added through SparkContext.addFile().

spark.apache.org

 

728x90
저작자표시 비영리

'공부' 카테고리의 다른 글

[Python] calling a function of a module by using its name (a string)  (0) 2020.10.03
[Java] forEach with index  (0) 2020.10.03
[Java] class field name to JSON  (0) 2020.10.03
[Python] Spark rdd foreach print  (0) 2020.10.03
[CSS] scrollbar visible  (0) 2020.10.03
댓글