Polars DataFrame quantile() Method

In Polars, the quantile() method is used to compute the quantile (or percentile) for each column in a DataFrame. It is especially useful for determining key percentiles, such as the median (50th percentile), or any other specified quantile. This function enables quick retrieval of specific quantile values (e.g., 25th, 50th, or 75th percentile) for each column. It’s useful for statistical analysis and understanding data distribution.

In this article, I will explain the quantile() method, including its syntax, parameters, and usage. Through detailed examples, I will demonstrate how to generate a DataFrame with the computed quantiles for the specified columns.

Key Points –

The quantile() method computes the specified quantile(s) for numerical columns in a Polars DataFrame.
The quantile parameter accepts a float between 0 and 1 (e.g., 0.25 for the first quartile, 0.5 for the median).
The default interpolation method is 'nearest', which selects the nearest value when the quantile falls between two data points.
You can use different interpolation methods, including 'linear', 'higher', 'lower', and 'midpoint', depending on how to handle quantiles between values.
The method can be applied to multiple columns simultaneously, calculating the quantile for each one.
The quantile is calculated for each column independently, meaning each column's distribution is considered separately.
The quantile() method is useful in descriptive statistics, summarizing data distribution and variability.
Missing values (None or NaN) are automatically excluded from the quantile computation.

Polars DataFrame.quantile() Introduction

Let’s know the syntax of the Polars DataFrame quantile() method.


# Syntax of quantile()
DataFrame.quantile(
    quantile: float,
    interpolation: RollingInterpolationMethod = 'nearest'
) -> DataFrame

Parameters of the Polars DataFrame.quantile()

Following are the parameters of the quantile() method.

quantile (float) - A number between 0 and 1 that specifies the quantile to compute.
- 0.25 for the 25th percentile (1st quartile),
- 0.5 for the 50th percentile (median),
- 0.75 for the 75th percentile (3rd quartile).
interpolation (optional, default: 'nearest') - The interpolation method to use when the quantile falls between two data points. Available options.
- 'nearest' - Choose the nearest value.
- 'linear' - Linear interpolation between the two closest points.
- 'higher' - Choose the higher of the two points.
- 'lower' - Choose the lower of the two points.
- 'midpoint' - Average the two closest points.

Return Value

This function returns a DataFrame containing the calculated quantiles for each column.

Usage of Polars DataFrame quantile() Method

The quantile() method in Polars is used to calculate the quantile (percentile) of each column in a DataFrame. It allows you to compute any desired quantile, such as the median, quartiles, or other percentiles.

First, let’s create a Polars DataFrame.


import polars as pl

# Creating a new Polars DataFrame
technologies = {
    "Student Names": ["Jenny", "Charle", "Veena"],
    "Age": [21, 28, 36],
    "Height": [140, 155, 130],
    "Weight": [40, 50, 35]
}

df = pl.DataFrame(technologies)
print("Original DataFrame:\n", df)

Yields below output.

You can compute the 25th percentile (first quartile) for all numerical columns in a Polars DataFrame using the quantile() method.


# Compute the 25th percentile (Q1)
result = df.quantile(0.25)
print("25th Percentile:\n", result)

In the above example, the 25th percentile (Q1) represents the value below which 25% of the data falls for each numerical column.

Compute the Median (50th Percentile)

Alternatively, you can compute the median (50th percentile) for all numerical columns in a Polars DataFrame using the quantile(0.5) method.


# Compute the 50th percentile (Median)
result = df.quantile(0.5)
print("Median (50th Percentile):\n", result)

# Output:
# Median (50th Percentile):
 shape: (1, 4)
┌───────────────┬──────┬────────┬────────┐
│ Student Names ┆ Age  ┆ Height ┆ Weight │
│ ---           ┆ ---  ┆ ---    ┆ ---    │
│ str           ┆ f64  ┆ f64    ┆ f64    │
╞═══════════════╪══════╪════════╪════════╡
│ null          ┆ 28.0 ┆ 140.0  ┆ 40.0   │
└───────────────┴──────┴────────┴────────┘

In the above example, the median (50th percentile) represents the middle value of each numerical column, providing a measure of central tendency.

Compute the 75th Percentile

You can compute the 75th percentile (third quartile) for all numerical columns in a Polars DataFrame using the quantile(0.75) method.


# Compute the 75th percentile (Q3)
Result = df.quantile(0.75)
print("75th Percentile:\n", Result)

# Output:
# 75th Percentile:
# shape: (1, 4)
┌───────────────┬──────┬────────┬────────┐
│ Student Names ┆ Age  ┆ Height ┆ Weight │
│ ---           ┆ ---  ┆ ---    ┆ ---    │
│ str           ┆ f64  ┆ f64    ┆ f64    │
╞═══════════════╪══════╪════════╪════════╡
│ null          ┆ 36.0 ┆ 155.0  ┆ 50.0   │
└───────────────┴──────┴────────┴────────┘

In the above example, the 75th (Q3) percentile represents the value below which 75% of the data falls for each numerical column.

Compute the 0th Percentile (Minimum Value)

You can compute the 0th percentile (minimum value) for all numerical columns in a Polars DataFrame using the quantile(0.0) method.


# Compute the 0th percentile (Minimum value)
Result = df.quantile(0.0)
print("0th Percentile (Minimum Value):\n", Result)

# Output:
# 0th Percentile (Minimum Value):
# shape: (1, 4)
┌───────────────┬──────┬────────┬────────┐
│ Student Names ┆ Age  ┆ Height ┆ Weight │
│ ---           ┆ ---  ┆ ---    ┆ ---    │
│ str           ┆ f64  ┆ f64    ┆ f64    │
╞═══════════════╪══════╪════════╪════════╡
│ null          ┆ 21.0 ┆ 130.0  ┆ 35.0   │
└───────────────┴──────┴────────┴────────┘

In the above example, the 0th percentile is equivalent to the minimum value in each numerical column.

Compute the 50th Percentile with Linear Interpolation

Similarly, you can compute the 50th percentile (median) with linear interpolation in polars using the quantile(0.5, "linear") method. The linear interpolation method estimates the quantile by linearly interpolating between data points when the exact quantile index falls between two values.


# Compute the 50th percentile (Median) with linear interpolation
Result = df.quantile(0.5, interpolation="linear")
print("Median (50th Percentile) with Linear Interpolation:\n", Result)

# Output:
# Median (50th Percentile) with Linear Interpolation:
# shape: (1, 4)
┌───────────────┬──────┬────────┬────────┐
│ Student Names ┆ Age  ┆ Height ┆ Weight │
│ ---           ┆ ---  ┆ ---    ┆ ---    │
│ str           ┆ f64  ┆ f64    ┆ f64    │
╞═══════════════╪══════╪════════╪════════╡
│ null          ┆ 28.0 ┆ 140.0  ┆ 40.0   │
└───────────────┴──────┴────────┴────────┘

Compute the 50th Percentile with Nearest Interpolation

Finally, you can compute the 50th percentile (median) with nearest interpolation in Polars using the quantile(0.5, interpolation="nearest") method. The nearest interpolation method selects the closest observed data point to the computed quantile index.


# Compute the 50th percentile (Median) with nearest interpolation
result = df.quantile(0.5, interpolation="nearest")
print("Median (50th Percentile) with Nearest Interpolation:\n", result)

# Output:
# Median (50th Percentile) with Nearest Interpolation:
# shape: (1, 4)
┌───────────────┬──────┬────────┬────────┐
│ Student Names ┆ Age  ┆ Height ┆ Weight │
│ ---           ┆ ---  ┆ ---    ┆ ---    │
│ str           ┆ f64  ┆ f64    ┆ f64    │
╞═══════════════╪══════╪════════╪════════╡
│ null          ┆ 28.0 ┆ 140.0  ┆ 40.0   │
└───────────────┴──────┴────────┴────────┘

Conclusion

In summary, the quantile() method in Polars is a valuable tool for statistical analysis, enabling the computation of specific percentiles such as the 0th (minimum), 25th, 50th (median), 75th, and 100th (maximum). It aids in understanding data distribution, identifying outliers, and efficiently summarizing key statistics.

Happy Learning!!

References

https://docs.pola.rs/api/python/stable/reference/dataframe/api/polars.DataFrame.product.html