Built with Tutify.io

    AIStudyLabs

    How to Update the Polars DataFrame

    How to Update the Polars DataFrame

    Updating a Polars DataFrame refers to modifying or transforming the data within a DataFrame by applying new values, operations, or conditions. Since Polars uses immutable DataFrames, updates don't modify the existing DataFrame directly. Instead, they return a new DataFrame with the desired changes applied.

    Updating a DataFrame typically involves performing various operations like mathematical transformations, string processing, filtering rows, adding new columns from existing data, modifying column values, renaming columns, or removing columns. In this article, I will explain how to update the polars DataFrame.

    Key Points –

    <a id="h-usage-of-polars-dataframe-update"></a>

    Usage of Polars DataFrame Update

    To update a column, use the with_columns() method to apply transformations to the column. Since Polars DataFrames are immutable, this operation will return a new DataFrame with the updated column.

    To run some examples of how to update the Polars DataFrame, let’s create a Polars DataFrame.

    
import polars as pl

technologies= {
    'Courses':["Spark", "PySpark", "Polars", "Pandas"],
    'Fees' :[22000, 25000, 30000, 35000],
    'Discount':[1000, 1500, 2500, 2000],
    'Duration':['30days','40days','50days','60days']
          }

df = pl.DataFrame(technologies)
print("Original DataFrame:\n", df)

    Yields below output.

    To update a single column in a Polars DataFrame, you can use the with_columns() method, which enables you to modify a specific column by creating a new version of it. For instance, if you want to increase the "Fees" by 10% for all courses, you can do it like this.

    
# Update the 'Fees' column by adding 10%
df2 = df.with_columns(
    (pl.col("Fees") * 1.10).alias("Fees")
)
print("Updated DataFrame:\n", df2)

    Here,

    <a id="h-create-a-new-column-based-on-existing-columns"></a>

    Create a New Column Based on Existing Columns

    To add a new column to a Polars DataFrame, you can use the with_columns() method along with a calculation or list of values that will populate the new column. For instance, let's say you want to add a new column called Final_Fees, which is calculated as Fees - Discount.

    
# Add a new column 'Final_Fees' by subtracting 'Discount' from 'Fees'
df2 = df.with_columns(
    (pl.col("Fees") - pl.col("Discount")).alias("Final_Fees")
)
print("DataFrame with New Column:\n", df2)

# Output:
# DataFrame with New Column:
# shape: (4, 5)
┌─────────┬───────┬──────────┬──────────┬────────────┐
│ Courses ┆ Fees  ┆ Discount ┆ Duration ┆ Final_Fees │
│ ---     ┆ ---   ┆ ---      ┆ ---      ┆ ---        │
│ str     ┆ i64   ┆ i64      ┆ str      ┆ i64        │
╞═════════╪═══════╪══════════╪══════════╪════════════╡
│ Spark   ┆ 22000 ┆ 1000     ┆ 30days   ┆ 21000      │
│ PySpark ┆ 25000 ┆ 1500     ┆ 40days   ┆ 23500      │
│ Polars  ┆ 30000 ┆ 2500     ┆ 50days   ┆ 27500      │
│ Pandas  ┆ 35000 ┆ 2000     ┆ 60days   ┆ 33000      │
└─────────┴───────┴──────────┴──────────┴────────────┘

    Here,

    <a id="h-update-multiple-columns-simultaneously"></a>

    Update Multiple Columns Simultaneously

    To update multiple columns simultaneously in Polars, you can pass multiple expressions to the with_columns() method. Each expression corresponds to a column that you want to update.

    Here's how to update both the Fees and Discount columns at the same time. For instance, let's increase the Fees by 10% and decrease the Discount by 5%.

    
# Update both 'Fees' and 'Discount' columns simultaneously
df2 = df.with_columns([
    (df['Fees'] * 1.10).alias('Fees'),
    (df['Discount'] - 100).alias('Discount')
])
print("DataFrame with Updated Columns:\n", df2)

# Output:
# DataFrame with Updated Columns:
# shape: (4, 4)
┌─────────┬─────────┬──────────┬──────────┐
│ Courses ┆ Fees    ┆ Discount ┆ Duration │
│ ---     ┆ ---     ┆ ---      ┆ ---      │
│ str     ┆ f64     ┆ i64      ┆ str      │
╞═════════╪═════════╪══════════╪══════════╡
│ Spark   ┆ 24200.0 ┆ 900      ┆ 30days   │
│ PySpark ┆ 27500.0 ┆ 1400     ┆ 40days   │
│ Polars  ┆ 33000.0 ┆ 2400     ┆ 50days   │
│ Pandas  ┆ 38500.0 ┆ 1900     ┆ 60days   │
└─────────┴─────────┴──────────┴──────────┘

    Here,

    <a id="h-apply-string-operations-to-update-a-column"></a>

    Apply String Operations to Update a Column

    To apply string operations in Polars, you can use various string methods available through the pl.col() method. If you want to update a column based on string manipulation, you can apply operations such as concatenation, trimming, or converting to uppercase/lowercase, etc.

    
# Apply string operation 
# To update the 'Courses' column (convert to uppercase)
df2 = df.with_columns(
    pl.col("Courses").str.to_uppercase().alias("Courses")
)
print("Updated DataFrame with Uppercase Courses:\n", df2)

# Output:
# Updated DataFrame with Uppercase Courses:
# shape: (4, 4)
┌─────────┬───────┬──────────┬──────────┐
│ Courses ┆ Fees  ┆ Discount ┆ Duration │
│ ---     ┆ ---   ┆ ---      ┆ ---      │
│ str     ┆ i64   ┆ i64      ┆ str      │
╞═════════╪═══════╪══════════╪══════════╡
│ SPARK   ┆ 22000 ┆ 1000     ┆ 30days   │
│ PYSPARK ┆ 25000 ┆ 1500     ┆ 40days   │
│ POLARS  ┆ 30000 ┆ 2500     ┆ 50days   │
│ PANDAS  ┆ 35000 ┆ 2000     ┆ 60days   │
└─────────┴───────┴──────────┴──────────┘

    Here,

    <a id="h-update-based-on-operations-involving-multiple-columns"></a>

    Update Based on Operations Involving Multiple Columns

    You can update a column in Polars based on operations involving multiple columns. For instance, if you want to update the Fees column based on a calculation involving both the Fees and Discount columns (e.g., subtracting the Discount from Fees and then adding a fixed amount), you can do this easily using the with_columns() method.

    
# Update 'Fees' column based on 'Fees' and 'Discount' columns
df2 = df.with_columns(
    ((df['Fees'] - df['Discount']) + 500).alias('Fees')
)
print("DataFrame with Updated 'Fees' Column:\n", df2)

# Output:
# DataFrame with Updated 'Fees' Column:
# shape: (4, 4)
┌─────────┬───────┬──────────┬──────────┐
│ Courses ┆ Fees  ┆ Discount ┆ Duration │
│ ---     ┆ ---   ┆ ---      ┆ ---      │
│ str     ┆ i64   ┆ i64      ┆ str      │
╞═════════╪═══════╪══════════╪══════════╡
│ Spark   ┆ 21500 ┆ 1000     ┆ 30days   │
│ PySpark ┆ 24000 ┆ 1500     ┆ 40days   │
│ Polars  ┆ 28000 ┆ 2500     ┆ 50days   │
│ Pandas  ┆ 33500 ┆ 2000     ┆ 60days   │
└─────────┴───────┴──────────┴──────────┘

    Here,

    <a id="h-rename-a-column"></a>

    Rename a Column

    To rename a column in a Polars DataFrame, you can use the rename() method, which allows you to specify the old column name and the new column name.

    
# Rename the 'Duration' column to 'Course_Duration'
df2 = df.rename({'Duration': 'Course_Duration'})
print("DataFrame with Renamed Column:\n", df2)

# Output:
# DataFrame with Renamed Column:
# shape: (4, 4)
┌─────────┬───────┬──────────┬─────────────────┐
│ Courses ┆ Fees  ┆ Discount ┆ Course_Duration │
│ ---     ┆ ---   ┆ ---      ┆ ---             │
│ str     ┆ i64   ┆ i64      ┆ str             │
╞═════════╪═══════╪══════════╪═════════════════╡
│ Spark   ┆ 22000 ┆ 1000     ┆ 30days          │
│ PySpark ┆ 25000 ┆ 1500     ┆ 40days          │
│ Polars  ┆ 30000 ┆ 2500     ┆ 50days          │
│ Pandas  ┆ 35000 ┆ 2000     ┆ 60days          │
└─────────┴───────┴──────────┴─────────────────┘

    Here,

    <a id="h-conclusion"></a>

    Conclusion

    In conclusion, updating a Polars DataFrame is a powerful and flexible operation that allows you to modify or create new columns using expressions. With methods like with_columns(), you can easily apply transformations, perform conditional updates, add new columns, and even rename existing ones.

    Happy Learning!!

    <a id="h-related-articles"></a>

    Related Articles

    <a id="h-reference"></a>

    Reference