티스토리 뷰
공부
[Java] pandas add filename column `df.withColumn("filename", input_file_name())
승가비 2022. 7. 31. 02:47728x90
import org.apache.spark.sql.functions.input_file_name
df.withColumn("filename", input_file_name())https://stackoverflow.com/questions/39868263/spark-load-data-and-add-filename-as-dataframe-column
Spark load data and add filename as dataframe column
I am loading some data into Spark with a wrapper function: def load_data( filename ): df = sqlContext.read.format("com.databricks.spark.csv")\ .option("delimiter", "\t")\ .opti...
stackoverflow.com
https://docs.databricks.com/sql/language-manual/functions/input_file_name.html
input_file_name function (Databricks SQL) | Databricks on AWS
docs.databricks.com
728x90
'공부' 카테고리의 다른 글
| [github] actions workflows `on.schedule.cron: * * * * *` delay (0) | 2022.07.31 |
|---|---|
| [Python] pandas `csv` -> `parquet` (0) | 2022.07.31 |
| [Python] pandas `df.reset_index()` (0) | 2022.07.31 |
| [Avro] Schema Registry Compatibility Level (0) | 2022.07.31 |
| [Kotlin] `*` vs `Any` (0) | 2022.07.31 |
댓글