Polars DataFrame std() – Usage & Examples

In Polars, the std() function is used to compute the standard deviation of numerical columns in a DataFrame. The standard deviation measures the dispersion or spread of data points relative to the mean and is useful for understanding the variability in your data.

In this article, I will explain the Polars DataFrame std() function and by using its syntax, parameters, and usage how we can return a new Polars DataFrame containing the standard deviation for each numeric column.

Key Points –

• The std() function calculates the standard deviation of numeric columns in a Polars DataFrame.
• By default, it computes the sample standard deviation (dividing by N-1).
• The default ddof=1 applies Bessel’s correction to provide an unbiased estimate of the standard deviation.
• Setting ddof=0 enables the calculation of the population standard deviation.
• Only applies to numerical columns; ignores categorical or string data.
• Polars automatically ignores null values while computing the standard deviation.
• Polars is optimized for performance, making std() faster than traditional pandas implementations.
• Can compute the standard deviation for specific columns using df.select(pl.col("column").std()).

<a id="h-polars-dataframe-std-introduction"></a>

Polars DataFrame std() Introduction

Let’s know the syntax of the std() function.

# Syntax of std()
DataFrame.std(ddof: int = 1) → DataFrame

<a id="h-parameters-of-the-polars-dataframe-std"></a>

Parameters of the Polars DataFrame.std()

It allows only one parameter.

• ddof - (int, optional, default = 1) Degrees of freedom for standard deviation calculation
  • 1 - Sample standard deviation (default, divides by n-1).
  • 0 - Population standard deviation (divides by n).

<a id="h-return-value"></a>

Return Value

This function returns a new Polars DataFrame with standard deviation values for each numeric column.

<a id="h-usage-of-polars-dataframe-std-function"></a>

Usage of Polars DataFrame std() Function

The std() function in Polars computes the standard deviation of numeric columns in a DataFrame. It measures the spread of data points from the mean.

First, let’s create a Polars DataFrame.

import polars as pl

# Creating a sample DataFrame
data = {
    'A': [15, 38, 13, 24],
    'B': [32, 21, 49, 11],
    'C': [12, 22, 36, 18]
}

df = pl.DataFrame(data)
print("Original DataFrame:\n",df)

Yields below output.

To compute the standard deviation for all numeric columns in a Polars DataFrame, use the std() function with its default setting (ddof=1). This calculates the sample standard deviation for each column while ignoring missing values.

# Compute standard deviation (default ddof=1)
std_df = df.std()
print("Standard Deviation:\n", std_df)

Here,

• df.std() computes the standard deviation for all numeric columns.
• The default ddof=1 applies Bessel’s correction (divides by n-1).
• The result is a new DataFrame with one row, containing the standard deviation of each column.

<a id="h-standard-deviation-with-ddof-0-population-std"></a>

Standard Deviation with ddof=0 (Population std)

By default, the std() function in Polars calculates the sample standard deviation (ddof=1). To compute the population standard deviation (dividing by n instead of n-1), set ddof=0. This involves first calculating the variance and then taking the square root.

# Compute population standard deviation (ddof=0)
std_df = df.std(ddof=0)
print("Population Standard Deviation:\n", std_df)

# Output:
# Population Standard Deviation:
# shape: (1, 3)
┌──────────┬───────────┬──────────┐
│ A        ┆ B         ┆ C        │
│ ---      ┆ ---       ┆ ---      │
│ f64      ┆ f64       ┆ f64      │
╞══════════╪═══════════╪══════════╡
│ 9.861541 ┆ 14.095655 ┆ 8.831761 │
└──────────┴───────────┴──────────┘

Here,

• df.std(ddof=0) computes the population standard deviation.
• ddof=0 means the divisor is n (total count) instead of n-1 (used in sample std).
• The result is a new DataFrame containing the population standard deviation for each numeric column.

<a id="h-standard-deviation-for-a-single-column"></a>

Standard Deviation for a Single Column

To compute the standard deviation for a single column in a Polars DataFrame, use select() function on that specific column. By default, it calculates the sample standard deviation (ddof=1).

# Compute standard deviation for column 'A' (default ddof=1)
std_df = df.select(pl.col("A").std())
print("Standard Deviation of Column 'A':\n", std_df)

# Output:
# Standard Deviation of Column 'A':
# shape: (1, 1)
┌───────────┐
│ A         │
│ ---       │
│ f64       │
╞═══════════╡
│ 11.387127 │
└───────────┘

Here,

• df.select(pl.col("A").std()) computes the standard deviation only for column "A".
• The default ddof=1 calculates the sample standard deviation.
• Returns a new DataFrame with a single column containing the standard deviation value.

<a id="h-standard-deviation-on-a-filtered-dataframe"></a>

Standard Deviation on a Filtered DataFrame

You can compute the standard deviation on a filtered Polars DataFrame by applying filter() before using std(). This allows you to calculate the standard deviation for a subset of rows that meet a specific condition.

# Filtering: Select rows where column 'A' > 20
filtered_df = df.filter(pl.col("A") > 20)

# Compute standard deviation for numeric columns 
# In filtered DataFrame (sample std, ddof=1)
std_df = filtered_df.std()
print("Filtered DataFrame:\n", filtered_df)
print("Standard Deviation on Filtered DataFrame:\n", std_df)

# Output:
# Filtered DataFrame:
# shape: (2, 3)
┌─────┬─────┬─────┐
│ A   ┆ B   ┆ C   │
│ --- ┆ --- ┆ --- │
│ i64 ┆ i64 ┆ i64 │
╞═════╪═════╪═════╡
│ 38  ┆ 21  ┆ 22  │
│ 24  ┆ 11  ┆ 18  │
└─────┴─────┴─────┘
# Standard Deviation on Filtered DataFrame:
# shape: (1, 3)
┌──────────┬──────────┬──────────┐
│ A        ┆ B        ┆ C        │
│ ---      ┆ ---      ┆ ---      │
│ f64      ┆ f64      ┆ f64      │
╞══════════╪══════════╪══════════╡
│ 9.899495 ┆ 7.071068 ┆ 2.828427 │
└──────────┴──────────┴──────────┘

Here,

• Filter rows where column A > 20 using df.filter(pl.col("A") > 20).
• Compute standard deviation on the filtered DataFrame using std().
• The filtered DataFrame has only selected rows, and the standard deviation is computed based on these values.

<a id="h-standard-deviation-with-missing-values"></a>

Standard Deviation with Missing Values

When computing the standard deviation in a Polars DataFrame that contains missing values (nulls), Polars automatically ignores NaN or null values during the calculation.

import polars as pl

# Creating a DataFrame with missing values
data = {
    'A': [15, 38, None, 24, 50],
    'B': [32, None, 49, 11, 40],
    'C': [None, 22, 36, 18, 45]
}

df = pl.DataFrame(data)

# Compute standard deviation (ignoring missing values)
std_df = df.std()

print("Original DataFrame with Missing Values:\n", df)
print("Standard Deviation (Ignoring NaN/Nulls):\n", std_df)

# Output:
# Original DataFrame with Missing Values:
# shape: (5, 3)
┌──────┬──────┬──────┐
│ A    ┆ B    ┆ C    │
│ ---  ┆ ---  ┆ ---  │
│ i64  ┆ i64  ┆ i64  │
╞══════╪══════╪══════╡
│ 15   ┆ 32   ┆ null │
│ 38   ┆ null ┆ 22   │
│ null ┆ 49   ┆ 36   │
│ 24   ┆ 11   ┆ 18   │
│ 50   ┆ 40   ┆ 45   │
└──────┴──────┴──────┘
# Standard Deviation (Ignoring NaN/Nulls):
# shape: (1, 3)
┌───────────┬───────────┬──────┐
│ A         ┆ B         ┆ C    │
│ ---       ┆ ---       ┆ ---  │
│ f64       ┆ f64       ┆ f64  │
╞═══════════╪═══════════╪══════╡
│ 15.413738 ┆ 16.227549 ┆ 12.5 │
└───────────┴───────────┴──────┘

Here,

• std() automatically ignores null values instead of treating them as 0.
• The calculation is performed only on available (non-null) values.
• If a column has all null values, std() will return null for that column.

<a id="h-conclusion"></a>

Conclusion

In summary, the std() function in Polars is an efficient tool for computing the standard deviation of numeric columns. It handles missing values seamlessly and allows control over sample (ddof=1) or population (ddof=0) calculations. Whether used on an entire DataFrame, a single column, or a filtered subset, it offers both flexibility and accuracy in statistical analysis.

Happy Learning!!

<a id="h-related-articles"></a>

Related Articles

• Polars DataFrame limit() Method
• Polars DataFrame describe() Function
• How to drop a column using Polars
• Polars Cast Multiple Columns
• How to Transpose DataFrame in Polars
• Polars DataFrame tail() – Usage & Examples
• Polars DataFrame update() – Usage & Examples
• Polars DataFrame fill_null() Usage & Examples

<a id="h-references"></a>

References

• https://docs.pola.rs/api/python/stable/reference/dataframe/api/polars.DataFrame.quantile.html