티스토리 뷰
728x90
from pyspark.sql.functions import lit
df = sqlContext.createDataFrame(
[(1, "a", 23.0), (3, "B", -23.0)], ("x1", "x2", "x3"))
df_with_x4 = df.withColumn("x4", lit(0))
df_with_x4.show()
## +---+---+-----+---+
## | x1| x2| x3| x4|
## +---+---+-----+---+
## | 1| a| 23.0| 0|
## | 3| B|-23.0| 0|
## +---+---+-----+---+How do I add a new column to a Spark DataFrame (using PySpark)?
I have a Spark DataFrame (using PySpark 1.5.1) and would like to add a new column. I've tried the following without any success: type(randomed_hours) # => list # Create in Python and transfor...
stackoverflow.com
728x90
'공부' 카테고리의 다른 글
| [Python] Split string every nth character? (0) | 2021.09.11 |
|---|---|
| [Linux] check DNS (0) | 2021.09.11 |
| [Java][Spark] add column & default (0) | 2021.09.11 |
| [Java][Spark] pyspark.sql.Row.asDict (0) | 2021.09.11 |
| [Java][Spark] rename column - df = df.withColumnRenamed("colName", "newColName") (0) | 2021.09.11 |
| [Java][Spark] new Row(**dict) (0) | 2021.09.11 |
댓글