How to Convert a Polars DataFrame to Python List?

In Polars, you can convert a DataFrame to a list using either the to_dicts(), rows(), or to_numpy() methods, depending on the desired output format. Polars DataFrame is a high-performance, efficient, and scalable tabular data structure built for data processing. While it functions similarly to a Pandas DataFrame, it is specifically optimized for speed and efficiency, particularly with large datasets and multi-threaded operations. In this article, I will explain how to convert a Polars DataFrame into a Python list.

Key Points –

Use to_dicts() followed by extracting values to get a list of lists.
Use to_list() on a Polars Series to extract values as a Python list.
Use rows() to obtain a list of tuples representing the DataFrame rows.
Convert a DataFrame to a List of Named Tuples by using df.rows(named=True) to enhance readability.
Use to_series().to_list() for each column to extract all data.
Convert a DataFrame into a list of dictionaries using to_dicts() to generate a structured row-wise dictionary representation.
rows() is faster than to_dicts() when working with large DataFrames.
Convert multiple columns to separate lists by selecting multiple columns and applying to_list() to each.

Usage of Convert Polars DataFrame to List

You can convert a DataFrame into a list using to_dicts() or to_numpy(), depending on the desired structure. This conversion is beneficial for data processing, transformation, and seamless integration with other libraries.

To run some examples of converting a Polars DataFrame to a Python list, let’s create a Polars DataFrame.


import polars as pl

technologies = {
    'Courses': ["Spark", "PySpark", "Python"],
    'Fee': [22000, 25000, 26000],
    'Duration': ['30days', '50days', '40days'],
    'Discount': [1000, 2300, 2500]
}

df = pl.DataFrame(technologies)
print("Create DataFrame:\n", df)

Yields below output.

You can convert a Polars DataFrame to a list of tuples using the rows() method. This method returns the rows of the DataFrame as a list of tuples, where each tuple represents a row.


# Convert to list of tuples
result = df.rows()
print("After converting to list of tuples:\n", result)

Here,

Create a Polars DataFrame from a dictionary.
The rows() method extracts the DataFrame row-wise and returns a list of tuples.
Each tuple contains all column values for that row, preserving the order.

Convert a Polars DataFrame to a List of Named Tuples

You can use rows(named=True) to convert a polars DataFrame into a list of named tuples. Named tuples allow you to access values using both index and column names.


# Convert DataFrame to a List of Named Tuples
result = df.rows(named=True)
print("After converting DataFrame to a list of named tuples:\n", result)

# Ouitput:
# After converting DataFrame to a list of named tuples:
# [{'Courses': 'Spark', 'Fee': 22000, 'Duration': '30days', 'Discount': 1000}, {'Courses': 'PySpark', 'Fee': 25000, 'Duration': '50days', 'Discount': 2300}, {'Courses': 'Python', 'Fee': 26000, 'Duration': '40days', 'Discount': 2500}]

Here,

The rows(named=True) method returns a list of named tuples instead of simple tuples.
Each named tuple contains the row values with column names attached.

Convert Entire DataFrame to a List of Lists

You can convert an entire polars DataFrame into a list of lists using the to_numpy().tolist() method or list comprehension.


# Convert DataFrame to a list of lists
result = df.to_numpy().tolist()
print("DataFrame as List of Lists:\n", result)

# Output:
# DataFrame as List of Lists:
 [['Spark', 22000, '30days', 1000], ['PySpark', 25000, '50days', 2300], ['Python', 26000, '40days', 2500]]

Here,

to_numpy() converts the Polars DataFrame to a NumPy-like array.
tolist() converts the array into a list of lists, where each inner list represents a row.

Convert a Single Column to a List

You can convert a single column from a Polars DataFrame to a list using the to_list() method.


# Convert a single column to a list
result = df['Courses'].to_list()
print("Courses column as List:\n", result)

# Output:
# Courses column as List:
# ['Spark', 'PySpark', 'Python']

Here,

df['Courses'] selects the 'Courses' column.
to_list() converts the Polars Series to a Python list.

Convert the Entire DataFrame to a List of Dictionaries

You can use to_dicts() to convert an entire DataFrame into a list of dictionaries, where each row is represented as a dictionary with column names as keys.


# Convert DataFrame to a list of dictionaries
result = df.to_dicts()
print("DataFrame as List of Dictionaries:\n", result)

# Output:
# DataFrame as List of Dictionaries:
# [{'Courses': 'Spark', 'Fee': 22000, 'Duration': '30days', 'Discount': 1000}, {'Courses': 'PySpark', 'Fee': 25000, 'Duration': '50days', 'Discount': 2300}, {'Courses': 'Python', 'Fee': 26000, 'Duration': '40days', 'Discount': 2500}]

Here,

to_dicts() converts each row of the Polars DataFrame into a dictionary, where keys are column names and values are the corresponding row values.
The output is a list of dictionaries, where each dictionary represents a row in the DataFrame.

Convert Each Column to a Separate List (Dictionary of Lists)

To convert each column into a separate Python list and store them in a dictionary, use the to_dict() method with as_series=False. This will generate a dictionary where the column names serve as keys, and the corresponding values are lists of column data.


# Convert each column to a separate list (Dictionary of Lists)
result = df.to_dict(as_series=False)
print(result)

# Output:
# {'Courses': ['Spark', 'PySpark', 'Python'], 'Fee': [22000, 25000, 26000], 'Duration': ['30days', '50days', '40days'], 'Discount': [1000, 2300, 2500]}

Here,

Each column in the DataFrame is converted into a list, forming a dictionary where: Keys are the column names. Values are lists containing column values.

Conclusion

In summary, converting a Pandas DataFrame to a Python list can be done in multiple ways, depending on the desired structure. Whether you need a list of lists, list of dictionaries, list of tuples, or individual column lists, Pandas provides efficient methods like df.rows(), values.tolist(), to_dict(), and itertuples() functions.

Happy Learning!!

References

https://docs.pola.rs/api/python/stable/reference/dataframe/api/polars.DataFrame.columns.html

How to Convert a Polars DataFrame to Python List?

Usage of Convert Polars DataFrame to List

Convert a Polars DataFrame to a List of Named Tuples

Convert Entire DataFrame to a List of Lists

Convert a Single Column to a List

Convert the Entire DataFrame to a List of Dictionaries

Convert Each Column to a Separate List (Dictionary of Lists)

Conclusion

Related Articles

References