공부
[Java] pandas add filename column `df.withColumn("filename", input_file_name())
승가비
2022. 7. 31. 02:47
728x90
import org.apache.spark.sql.functions.input_file_name
df.withColumn("filename", input_file_name())https://stackoverflow.com/questions/39868263/spark-load-data-and-add-filename-as-dataframe-column
Spark load data and add filename as dataframe column
I am loading some data into Spark with a wrapper function: def load_data( filename ): df = sqlContext.read.format("com.databricks.spark.csv")\ .option("delimiter", "\t")\ .opti...
stackoverflow.com
https://docs.databricks.com/sql/language-manual/functions/input_file_name.html
input_file_name function (Databricks SQL) | Databricks on AWS
docs.databricks.com
728x90