Add a New Column into an Existing Polars DataFrame

In Polars, to add a new column to an existing DataFrame, you can use the with_columns() method. This method enables you to add one or more new columns, which can either be derived from existing data or based on custom values. Adding a column means creating and appending a new column to a DataFrame that already contains other data. Note that this operation does not alter the original DataFrame directly; instead, it returns a new DataFrame that includes the newly added column. In this article, I will explain how the add a new column to an existing Polars DataFrame.

Key Points –

The with_columns() method is commonly used to add one or more new columns to a DataFrame.
You can add a column with a constant value by using pl.lit() to create a constant expression.
Adding a new column doesn't modify the original DataFrame; it returns a new one.
You can use insert_column() to add a new column at a specific position.
Polars supports arithmetic, logical, and conditional operations for column transformations.
Add new columns by applying operations or functions on existing columns in the DataFrame.
Polars allows you to perform rolling operations (e.g., moving averages) and add the result as a new column.
You can add a new column by creating a pl.Series of values, ensuring the length matches the number of rows in the DataFrame.
When inserting a new column, you can use alias() to name the column appropriately.

Usage of Add a New Column into an Existing Polars DataFrame

To add a new column to an existing Polars DataFrame, you can use the with_columns() method. This allows you to add one or more columns based on expressions, existing columns, or constant values.

To run some examples of how to add a new column to an existing polars DataFrame, let’s create a Polars DataFrame.


import polars as pl

studentdetails = {
       "Studentname":["Ram", "Sam", "Scott", "Ann", "John"],
       "Mathematics" :[80,90,85,72,95],
       "Science" :[85,95,80,90,92],
              }
df = pl.DataFrame(studentdetails)
print("Original DataFrame:\n", df)

Yields below output.

To add a constant value column in Polars, you can use the with_columns() method along with pl.lit() to create a literal (constant) value for all rows. This allows you to easily insert a column with the same constant value across the entire DataFrame.


# Add constant value column "Grade" with value "A"
df2 = df.with_columns(pl.lit("A").alias("Grade"))
print("DataFrame with constant column:\n", df2)

Yields below output.

Adding a Column by Performing a Simple Operation on an Existing Column

To add a new column by performing a simple operation on an existing column in Polars, you use the with_columns() method along with an expression. Here's how you can add a new column that is the result of an operation on the existing columns.


# Add new column by performing operation on existing column
df2 = df.with_columns((pl.col("Mathematics") + 5).alias("Math_with_bonus"))
print("DataFrame with new computed column:\n", df2)

# Output:
# DataFrame with new computed column:
 shape: (5, 4)
┌─────────────┬─────────────┬─────────┬─────────────────┐
│ Studentname ┆ Mathematics ┆ Science ┆ Math_with_bonus │
│ ---         ┆ ---         ┆ ---     ┆ ---             │
│ str         ┆ i64         ┆ i64     ┆ i64             │
╞═════════════╪═════════════╪═════════╪═════════════════╡
│ Ram         ┆ 80          ┆ 85      ┆ 85              │
│ Sam         ┆ 90          ┆ 95      ┆ 95              │
│ Scott       ┆ 85          ┆ 80      ┆ 90              │
│ Ann         ┆ 72          ┆ 90      ┆ 77              │
│ John        ┆ 95          ┆ 92      ┆ 100             │
└─────────────┴─────────────┴─────────┴─────────────────┘

Adding a Column Based on Multiple Columns

To add a new column based on multiple existing columns in Polars, you can apply arithmetic operations (such as addition, subtraction, or multiplication) or use conditional logic that involves more than one column. Just like single-column transformations, Polars makes it easy to combine or compare multiple columns using expressions within with_columns().


# Add a new column based on multiple existing columns
df2 = df.with_columns(
    ((df["Mathematics"] + df["Science"]) / 2).alias("AverageMarks")
)
print("DataFrame with AverageMarks Column:\n", df2)

# Output:
# DataFrame with AverageMarks Column:
# shape: (5, 4)
┌─────────────┬─────────────┬─────────┬──────────────┐
│ Studentname ┆ Mathematics ┆ Science ┆ AverageMarks │
│ ---         ┆ ---         ┆ ---     ┆ ---          │
│ str         ┆ i64         ┆ i64     ┆ f64          │
╞═════════════╪═════════════╪═════════╪══════════════╡
│ Ram         ┆ 80          ┆ 85      ┆ 82.5         │
│ Sam         ┆ 90          ┆ 95      ┆ 92.5         │
│ Scott       ┆ 85          ┆ 80      ┆ 82.5         │
│ Ann         ┆ 72          ┆ 90      ┆ 81.0         │
│ John        ┆ 95          ┆ 92      ┆ 93.5         │
└─────────────┴─────────────┴─────────┴──────────────┘

Inserting Column at a Specific Position

To insert a column at a specific position in a Polars DataFrame, you can use the insert_column() method, where the first argument is the index at which you want to insert the column, and the second argument is the column data you want to insert.


# Create a new column "English"
data = pl.Series([78, 88, 82, 85, 90])

# Insert the "English" column at the second position (index 1)
df2 = df.insert_column(1, data.alias("English"))
print("DataFrame with English inserted at position 1:\n", df2)

# Output:
# DataFrame with English inserted at position 1:
# shape: (5, 4)
┌─────────────┬─────────┬─────────────┬─────────┐
│ Studentname ┆ English ┆ Mathematics ┆ Science │
│ ---         ┆ ---     ┆ ---         ┆ ---     │
│ str         ┆ i64     ┆ i64         ┆ i64     │
╞═════════════╪═════════╪═════════════╪═════════╡
│ Ram         ┆ 78      ┆ 80          ┆ 85      │
│ Sam         ┆ 88      ┆ 90          ┆ 95      │
│ Scott       ┆ 82      ┆ 85          ┆ 80      │
│ Ann         ┆ 85      ┆ 72          ┆ 90      │
│ John        ┆ 90      ┆ 95          ┆ 92      │
└─────────────┴─────────┴─────────────┴─────────┘

Adding a Column with a Rolling Operation

Adding a column using a rolling operation in Polars involves performing a sliding window calculation on an existing column and saving the result as a new column. This is especially useful for time series or trend analysis, where you might compute moving averages, rolling sums, or other aggregations to smooth data or identify patterns over time.


# Add a rolling mean column with window size 3
df2 = df.with_columns(
    pl.col("Mathematics").rolling_mean(window_size=3).alias("Math_RollingAvg")
)
print("DataFrame with rolling mean column:\n", df2)

# Output:
# DataFrame with rolling mean column:
# shape: (5, 4)
┌─────────────┬─────────────┬─────────┬─────────────────┐
│ Studentname ┆ Mathematics ┆ Science ┆ Math_RollingAvg │
│ ---         ┆ ---         ┆ ---     ┆ ---             │
│ str         ┆ i64         ┆ i64     ┆ f64             │
╞═════════════╪═════════════╪═════════╪═════════════════╡
│ Ram         ┆ 80          ┆ 85      ┆ null            │
│ Sam         ┆ 90          ┆ 95      ┆ null            │
│ Scott       ┆ 85          ┆ 80      ┆ 85.0            │
│ Ann         ┆ 72          ┆ 90      ┆ 82.333333       │
│ John        ┆ 95          ┆ 92      ┆ 84.0            │
└─────────────┴─────────────┴─────────┴─────────────────┘

Adding a Column with a Series of Values

To add a column with a series of specific values to a Polars DataFrame, you can create a list or array of values and use the with_columns() method to assign it to a new column. Adding such a column is simple: just create a pl.Series with the desired values and add it to your DataFrame using either with_columns() or insert_column().


# Create a Series of grades
grades = pl.Series("Grade", ["B", "A", "A", "C", "A+"])

# Add the series as a new column
df2 = df.with_columns(grades)
print("DataFrame with Grades column:\n", df2)

# Output:
# DataFrame with Grades column:
# shape: (5, 4)
┌─────────────┬─────────────┬─────────┬───────┐
│ Studentname ┆ Mathematics ┆ Science ┆ Grade │
│ ---         ┆ ---         ┆ ---     ┆ ---   │
│ str         ┆ i64         ┆ i64     ┆ str   │
╞═════════════╪═════════════╪═════════╪═══════╡
│ Ram         ┆ 80          ┆ 85      ┆ B     │
│ Sam         ┆ 90          ┆ 95      ┆ A     │
│ Scott       ┆ 85          ┆ 80      ┆ A     │
│ Ann         ┆ 72          ┆ 90      ┆ C     │
│ John        ┆ 95          ┆ 92      ┆ A+    │
└─────────────┴─────────────┴─────────┴───────┘

Conclusion

In conclusion, adding new columns to a Polars DataFrame is a powerful way to enrich and transform your data. Whether you're performing simple operations like adding constant values, applying mathematical or conditional transformations, or handling more complex tasks such as rolling windows or combining multiple columns, Polars offers flexible methods to efficiently manage your data. The with_columns() and insert_column() methods give you full control over the placement and addition of new columns.

Happy Learning!!

Reference

https://docs.pola.rs/api/python/stable/reference/dataframe/api/polars.DataFrame.vstack.html