티스토리 뷰

공부

[Java][Spark] add column & default

승가비 2021. 9. 11. 08:46
728x90
from pyspark.sql.functions import lit

df = sqlContext.createDataFrame(
    [(1, "a", 23.0), (3, "B", -23.0)], ("x1", "x2", "x3"))

df_with_x4 = df.withColumn("x4", lit(0))
df_with_x4.show()

## +---+---+-----+---+
## | x1| x2|   x3| x4|
## +---+---+-----+---+
## |  1|  a| 23.0|  0|
## |  3|  B|-23.0|  0|
## +---+---+-----+---+

https://stackoverflow.com/questions/33681487/how-do-i-add-a-new-column-to-a-spark-dataframe-using-pyspark

 

How do I add a new column to a Spark DataFrame (using PySpark)?

I have a Spark DataFrame (using PySpark 1.5.1) and would like to add a new column. I've tried the following without any success: type(randomed_hours) # => list # Create in Python and transfor...

stackoverflow.com

 

728x90
저작자표시비영리

'공부' 카테고리의 다른 글

[Python] Split string every nth character?  (0) 2021.09.11
[Linux] check DNS  (0) 2021.09.11
[Java][Spark] add column & default  (0) 2021.09.11
[Java][Spark] pyspark.sql.Row.asDict  (0) 2021.09.11
[Java][Spark] rename column - df = df.withColumnRenamed("colName", "newColName")  (0) 2021.09.11
[Java][Spark] new Row(**dict)  (0) 2021.09.11
댓글
댓글쓰기 폼
1 ··· 342 343 344 345 346 347 348 349 350 ··· 1133