Built with Tutify.io

    AIStudyLabs

    How to Convert Struct to Series in Polars?

    How to Convert Struct to Series in Polars?

    In Polars, a Struct column acts like a single column containing multiple named fields, similar to a dictionary in each row. To convert a Struct column into individual Series (i.e., separate columns),

    Converting a Struct to Series in Polars refers to the process of transforming a Struct column, which contains multiple fields (often resembling a dictionary or nested structure), into a Series (a single column of data). This transformation makes it easier to work with, access, and manipulate the individual data points stored within the Struct. In this article, I will explain how to convert a struct to a series in polars.

    Key Points –

    <a id="h-what-is-a-struct-column-in-polars"></a>

    What is a Struct Column in Polars

    A Struct column in Polars is a type of column that holds complex data structures. Each row in the column contains a nested structure with multiple fields (similar to a dictionary or record). It allows you to group related values together within a single column, making it easier to handle complex data types.

    <a id="h-usage-of-convert-struct-to-series-in-polars"></a>

    Usage of Convert Struct to Series in Polars

    Converting a Struct column to a Series in Polars means flattening the Struct so that each individual field within the Struct becomes part of a long Series (instead of remaining nested in a column).

    Now, let’s create a Polars DataFrame.

    
import polars as pl

df = pl.DataFrame({
    "technologies": [{"Courses": "Spark", "Fees": 22000, "Discount":1000}]
})
print("Original DataFrame:\n", df)

    Yields below output.

    You can extract all fields from a Struct into separate columns at once in Polars by using the unnest() method. This method automatically expands the Struct column into individual fields as separate columns in the DataFrame.

    
# Unnest the struct column
unnested_df = df.unnest("technologies")
print("Unnested DataFrame:\n", unnested_df)

    Here,

    <a id="h-using-select-unnest"></a>

    Using select().unnest()

    To extract the fields from a Struct column using select().unnest() in Polars, you can combine these methods when you want to target specific columns and flatten the struct in a single step. This approach allows you to work directly with the struct field and unnest it, without affecting other columns in the DataFrame.

    
# Use select + unnest
df_unnested = df.select(pl.col("technologies")).unnest("technologies")
print("DataFrame after .select().unnest():\n", df_unnested)

# Output:
# DataFrame after .select().unnest():
 shape: (1, 3)
┌─────────┬───────┬──────────┐
│ Courses ┆ Fees  ┆ Discount │
│ ---     ┆ ---   ┆ ---      │
│ str     ┆ i64   ┆ i64      │
╞═════════╪═══════╪══════════╡
│ Spark   ┆ 22000 ┆ 1000     │
└─────────┴───────┴──────────┘

    <a id="h-using-select-unpivot"></a>

    Using select().unpivot()

    To use select().unpivot() in Polars to convert a Struct column into a long format, you can follow these steps. This method is particularly useful when you want to transform the data from a wide format (multiple columns) to a long format (key-value pairs).

    To preserve the original field names when converting a Struct column in Polars, you can use the unpivot() method without dropping the variable column. This will keep the field names as part of the resulting DataFrame.

    
# Unnest the struct column and then unpivot to convert into long format
df2 = df.select("technologies").unnest("technologies").unpivot(value_name="value")
print("Unpivoted DataFrame:\n", df2)

# Output:
# Unpivoted DataFrame:
# shape: (3, 2)
┌──────────┬───────┐
│ variable ┆ value │
│ ---      ┆ ---   │
│ str      ┆ str   │
╞══════════╪═══════╡
│ Courses  ┆ Spark │
│ Fees     ┆ 22000 │
│ Discount ┆ 1000  │
└──────────┴───────┘

    Here,

    <a id="h-using-select-unpivot-drop-variable"></a>

    Using select().unpivot().drop("variable")

    To use select().unpivot().drop("variable") in Polars, you can extract the fields from a Struct column, convert them into a long format, and then drop the column that holds the field names (variable), leaving only the values. This is useful when you want to keep just the values after unpivoting and discard the column that identifies the field names.

    
# Unnest the struct column, unpivot it, and drop the "variable" column
df2 = df.select("technologies").unnest("technologies").unpivot(value_name="value").drop("variable")
print("Unpivoted DataFrame without 'variable' column:\n", df2)

# Output:
# Unpivoted DataFrame without 'variable' column:
# shape: (3, 1)
┌───────┐
│ value │
│ ---   │
│ str   │
╞═══════╡
│ Spark │
│ 22000 │
│ 1000  │
└───────┘

    Here,

    <a id="h-using-transpose"></a>

    Using transpose()

    To transpose struct fields into rows in Polars, meaning that you want to take the fields from a Struct column and convert them into separate rows, you can follow a multi-step process. This involves first unnesting the struct into columns and then transposing those columns into rows.

    
# Unnest the 'technologies' column and transpose the DataFrame
df2 = df.unnest('technologies').transpose(column_names=['technologies'])
print(df2)

# Output:
# shape: (3, 1)
┌──────────────┐
│ technologies │
│ ---          │
│ str          │
╞══════════════╡
│ Spark        │
│ 22000        │
│ 1000         │
└──────────────┘

    Here,

    <a id="h-conclusion"></a>

    Conclusion

    In conclusion, converting Struct columns to Series in Polars is a powerful way to simplify and manipulate nested data. A Struct column can be thought of as a single column containing multiple fields, similar to a dictionary. Converting these fields into individual Series allows for better data access and flexibility.

    Happy Learning!!

    <a id="h-related-articles"></a>

    Related Articles

    <a id="h-reference"></a>

    Reference