Efficient way to Update a Single Element of a Polars DataFrame?

In Polars, the most efficient way to update a single element in a DataFrame is by using the with_columns() method along with a when-then-otherwise expression to apply conditional logic. Unlike pandas, Polars doesn't support in-place modifications, so updates are done by creating a new DataFrame through expression-based transformations.

Since Polars is a columnar, immutable DataFrame library, it doesn't support in-place mutation of individual elements like Pandas. Instead, it uses expression-based transformations. In this article, I will explain the efficient way to update a single element of a polars DataFrame.

Key Points –

Polars DataFrames are immutable, so updates create a new DataFrame instead of modifying the original in-place.
Use the with_columns() method to apply column transformations based on conditions.
Use expressions such as pl.when().then().otherwise() to describe the conditions for updates, ensuring clarity and performance.
You can apply updates based on column values, row indices, or combinations of multiple conditions.
You can combine multiple conditions using logical operators (& for "and", | for "or").
Polars works efficiently with column-based operations, so even for single element updates, ensure to use column expressions instead of row-wise operations.
Update entire columns or specific rows based on column conditions rather than manipulating individual elements directly.
Unlike Pandas, in Polars, there’s no in-place modification (e.g., no df.iloc[] equivalent), so always return the updated DataFrame.

Usage of Update a Single Element of a Polars DataFrame

To update a single element in a Polars DataFrame, you cannot directly modify individual elements like in pandas. Instead, use the with_columns() method combined with pl.when().then().otherwise() to apply a condition and update the desired column(s).

To run some examples of efficient way to update a single element of a polars DataFrame, let’s create a Polars DataFrame.


import polars as pl

technologies = {
    'Courses': ["Spark", "PySpark", "Hadoop", "Pandas"],
    'Fee': [22000, 25000, 24000, 26000],
    'Discount': [1000, 1200, 2500, 2000],
    'Duration': ['35days', '40days', '65days', '50days']
}

df = pl.DataFrame(technologies)
print("Original DataFrame:\n", df)

Yields below output.

To update a numeric column in your Polars DataFrame based on a condition, you can use the with_columns() method combined with a conditional expression. Let's say you want to update the "Fee" column for a specific course where the Fee is greater than 24000 and set those values to 30000.


# Update 'Fee' where the fee is greater than 24000
df2 = df.with_columns(
    pl.when(pl.col("Fee") > 24000)
      .then(30000)
      .otherwise(pl.col("Fee"))
      .alias("Fee")
)
print("Updated DataFrame:\n", df2)

Here,

pl.when(pl.col("Fee") > 24000): creates a condition where the "Fee" is greater than 24000.
then(30000): sets the "Fee" to 30000 for the rows where the condition is met.
otherwise(pl.col("Fee")): keeps the existing "Fee" values where the condition is not met.
alias("Fee"): ensures that the updated column is named "Fee".

Update a Numeric Column where a Condition Matches

To update a numeric column in a Polars DataFrame where a specific condition matches, you can use the with_columns() method with the when-then-otherwise expression.


# Update the 'Fee' for the 'PySpark' course
df2 = df.with_columns(
    pl.when(pl.col("Courses") == "PySpark")
    .then(28000)
    .otherwise(pl.col("Fee"))
    .alias("Fee")
)
print("Updated DataFrame:\n", df2)

# Output:
# Updated DataFrame:
# shape: (4, 4)
┌─────────┬───────┬──────────┬──────────┐
│ Courses ┆ Fee   ┆ Discount ┆ Duration │
│ ---     ┆ ---   ┆ ---      ┆ ---      │
│ str     ┆ i64   ┆ i64      ┆ str      │
╞═════════╪═══════╪══════════╪══════════╡
│ Spark   ┆ 22000 ┆ 1000     ┆ 35days   │
│ PySpark ┆ 28000 ┆ 1200     ┆ 40days   │
│ Hadoop  ┆ 24000 ┆ 2500     ┆ 65days   │
│ Pandas  ┆ 26000 ┆ 2000     ┆ 50days   │
└─────────┴───────┴──────────┴──────────┘

In the above example, the Fee column for the "PySpark" course is updated to 28000, and the other values remain the same.

Update Based on a Combination of Columns

To update a column based on a combination of conditions across multiple columns, you can combine conditions with logical operators (& for "and", | for "or") in Polars. Here's how you can update the "Fee" column based on a combination of two conditions. for example, if the "Courses" column is "PySpark" and the "Discount" is greater than 1000, update the "Fee" to 40000.


# Update 'Fee' based on combination of 'Courses' and 'Discount'
df2 = df.with_columns(
    pl.when((pl.col("Courses") == "PySpark") & (pl.col("Discount") > 1000))
      .then(40000)
      .otherwise(pl.col("Fee"))
      .alias("Fee")
)
print("Updated DataFrame:\n", df2)

# Output:
# Updated DataFrame:
# shape: (4, 4)
Updated DataFrame:
 shape: (4, 4)
┌─────────┬───────┬──────────┬──────────┐
│ Courses ┆ Fee   ┆ Discount ┆ Duration │
│ ---     ┆ ---   ┆ ---      ┆ ---      │
│ str     ┆ i64   ┆ i64      ┆ str      │
╞═════════╪═══════╪══════════╪══════════╡
│ Spark   ┆ 22000 ┆ 1000     ┆ 35days   │
│ PySpark ┆ 40000 ┆ 1200     ┆ 40days   │
│ Hadoop  ┆ 24000 ┆ 2500     ┆ 65days   │
│ Pandas  ┆ 26000 ┆ 2000     ┆ 50days   │
└─────────┴───────┴──────────┴──────────┘

Here,

(pl.col("Courses") == "PySpark") & (pl.col("Discount") > 1000): combines two conditions using the logical AND (&) operator. The "Fee" will be updated if both conditions are true.
then(40000): sets the "Fee" to 40000 where the conditions match.
otherwise(pl.col("Fee")): keeps the original "Fee" value where the conditions do not match.
alias("Fee"): ensures the updated column is still named "Fee".

Update Based on String Column Match

To update a column in a Polars DataFrame based on a string column match, you can use the with_columns() method along with pl.when().then().otherwise().


# Update Discount where Courses == "Pandas"
df2 = df.with_columns(
    pl.when(pl.col("Courses") == "Pandas")
    .then(1800)
    .otherwise(pl.col("Discount"))
    .alias("Discount")
)
print("Updated DataFrame:\n", df2)

# Output:
# Updated DataFrame:
# shape: (4, 4)
┌─────────┬───────┬──────────┬──────────┐
│ Courses ┆ Fee   ┆ Discount ┆ Duration │
│ ---     ┆ ---   ┆ ---      ┆ ---      │
│ str     ┆ i64   ┆ i64      ┆ str      │
╞═════════╪═══════╪══════════╪══════════╡
│ Spark   ┆ 22000 ┆ 1000     ┆ 35days   │
│ PySpark ┆ 25000 ┆ 1200     ┆ 40days   │
│ Hadoop  ┆ 24000 ┆ 2500     ┆ 65days   │
│ Pandas  ┆ 26000 ┆ 1800     ┆ 50days   │
└─────────┴───────┴──────────┴──────────┘

Here,

Match string value using pl.col("Courses") == "Pandas".
Use then() to set the new value.
Use otherwise() to retain existing values.

Update to a Computed Value

To assign a computed value to a column in Polars, you can use the with_columns() method along with expressions that perform calculations using other columns or constants.


# Update Fee to Fee * 1.10 where Courses == "Spark"
df2 = df.with_columns(
    pl.when(pl.col("Courses") == "Spark")
    .then(pl.col("Fee") * 1.10)
    .otherwise(pl.col("Fee"))
    .alias("Fee")
)
print("Updated DataFrame:\n", df2)

# Output:
# Updated DataFrame:
# shape: (4, 4)
┌─────────┬─────────┬──────────┬──────────┐
│ Courses ┆ Fee     ┆ Discount ┆ Duration │
│ ---     ┆ ---     ┆ ---      ┆ ---      │
│ str     ┆ f64     ┆ i64      ┆ str      │
╞═════════╪═════════╪══════════╪══════════╡
│ Spark   ┆ 24200.0 ┆ 1000     ┆ 35days   │
│ PySpark ┆ 25000.0 ┆ 1200     ┆ 40days   │
│ Hadoop  ┆ 24000.0 ┆ 2500     ┆ 65days   │
│ Pandas  ┆ 26000.0 ┆ 2000     ┆ 50days   │
└─────────┴─────────┴──────────┴──────────┘

Here,

pl.col("Fee") * 1.10: computes a 10% increase dynamically.
This works seamlessly for any arithmetic or transformation you want to apply.

Update a Boolean Column at a Specific Row

To update a boolean column at a specific row in a Polars DataFrame, you typically use with_columns() combined with when, then, and otherwise, using row conditions like row number or other column values.


import polars as pl

# Sample DataFrame
df = pl.DataFrame({
    'Courses': ["Spark", "PySpark", "Hadoop", "Pandas"],
    'IsActive': [True, True, True, True]
})

# Update IsActive to False at row index 2
df2 = df.with_columns(
    pl.when(pl.arange(0, df.height) == 2)
    .then(False)
    .otherwise(pl.col("IsActive"))
    .alias("IsActive")
)
print("Updated DataFrame:\n", df2)

# Output:
# Updated DataFrame:
# shape: (4, 2)
┌─────────┬──────────┐
│ Courses ┆ IsActive │
│ ---     ┆ ---      │
│ str     ┆ bool     │
╞═════════╪══════════╡
│ Spark   ┆ true     │
│ PySpark ┆ true     │
│ Hadoop  ┆ false    │
│ Pandas  ┆ true     │
└─────────┴──────────┘

Here,

pl.arange(0, df.height) creates a sequence [0, 1, 2, 3].
The condition == 2 targets the third row (0-based index).
then(False) updates that row to False.
otherwise(pl.col("IsActive")) keeps the original values elsewhere.

Conclusion

In summary, Polars provides a flexible and efficient way to update DataFrame columns using the with_columns() method. Whether you're applying conditional logic with pl.when().then().otherwise(), combining multiple conditions using logical operators, or computing new values from existing data, Polars makes these operations straightforward and performant.

Happy Learning!!

Reference

https://docs.pola.rs/api/python/stable/reference/dataframe/api/polars.DataFrame.upsample.html

Efficient way to Update a Single Element of a Polars DataFrame?

Usage of Update a Single Element of a Polars DataFrame

Update a Numeric Column where a Condition Matches

Update Based on a Combination of Columns

Update Based on String Column Match

Update to a Computed Value

Update a Boolean Column at a Specific Row

Conclusion

Related Articles

Reference