티스토리 뷰

공부 (@Deprecated)

[Java] pandas add filename column `df.withColumn("filename", input_file_name())

승가비 2022. 7. 31. 02:47
728x90
import org.apache.spark.sql.functions.input_file_name

df.withColumn("filename", input_file_name())

https://stackoverflow.com/questions/39868263/spark-load-data-and-add-filename-as-dataframe-column

 

Spark load data and add filename as dataframe column

I am loading some data into Spark with a wrapper function: def load_data( filename ): df = sqlContext.read.format("com.databricks.spark.csv")\ .option("delimiter", "\t")\ .opti...

stackoverflow.com

https://docs.databricks.com/sql/language-manual/functions/input_file_name.html

 

input_file_name function (Databricks SQL) | Databricks on AWS

 

docs.databricks.com

 

728x90
저작자표시 비영리 (새창열림)

'공부 (@Deprecated)' 카테고리의 다른 글

[github] actions workflows `on.schedule.cron: * * * * *` delay  (0) 2022.07.31
[Python] pandas `csv` -> `parquet`  (0) 2022.07.31
[Python] pandas `df.reset_index()`  (0) 2022.07.31
[Avro] Schema Registry Compatibility Level  (0) 2022.07.31
[Kotlin] `*` vs `Any`  (0) 2022.07.31
댓글