CodeFiori

Appearance

On this page

Renaming Columns in a DataFrame - .withColumnRenamed() ​

Overview ​

The withColumnRenamed() function is used to rename columns in a pyspark DataFrame. It allows you to change the name of one or more columns in the DataFrame while keeping the data and structure intact. The withColumnRenamed() function returns a new DataFrame with the renamed columns.

Rename a Single Column ​

You can use the withColumnRenamed() function to rename a single column in the DataFrame by providing the current column name and the desired new column name.

python
from pyspark.sql import SparkSession

# Create a SparkSession (if not already created)
spark = SparkSession.builder.appName("WithColumnRenamedExample").getOrCreate()

# Sample data as a list of dictionaries
data = [
    {"name": "Alice", "age": 30},
    {"name": "Bob", "age": 25},
    {"name": "Charlie", "age": 35},
]

# Create a DataFrame
df = spark.createDataFrame(data)

# Rename a single column
df_renamed = df.withColumnRenamed("age", "years_old")

df_renamed.show()

Output:

+-------+---------+
|   name|years_old|
+-------+---------+
|  Alice|       30|
|    Bob|       25|
|Charlie|       35|
+-------+---------+

Rename using select ​

you can use the alias() function in combination with the select() method to rename columns in a DataFrame. The alias() function allows you to provide an alternative name for a column, effectively renaming it in the resulting DataFrame.

python
from pyspark.sql import SparkSession
from pyspark.sql.functions import col

# Create a SparkSession (if not already created)
spark = SparkSession.builder.appName("AliasExample").getOrCreate()

# Sample data as a list of dictionaries
data = [{"name": "Alice", "age": 30},
        {"name": "Bob", "age": 25},
        {"name": "Charlie", "age": 35}]

# Create a DataFrame
df = spark.createDataFrame(data)

# Use 'alias()' to rename multiple columns in one select statement
renamed_df = df.select(col("name").alias("full_name"), col("age").alias("years_old"))


renamed_df.show()

Output:

+---------+---------+
|full_name|years_old|
+---------+---------+
|    Alice|       30|
|      Bob|       25|
|  Charlie|       35|
+---------+---------+