Explain PySpark sample() Function with examples

In PySpark, the sample() function is used to perform simple random sampling on a DataFrame. Unlike stratified sampling, simple random sampling selects rows randomly across the entire dataset without considering groupings.

For example, if you have a dataset of 1,000 customers, you might want to randomly pick 10% of the rows for testing purposes.

In simple terms, sample() randomly picks a subset of rows from the DataFrame according to the fraction you specify.

In this article, I will walk you through how sample() works with clear examples.

Key Points-

Used for simple random sampling in PySpark.
Samples rows across the entire DataFrame without considering groups.
Can sample with or without replacement.
Fraction specifies the approximate portion of the DataFrame to sample.
The original DataFrame remains unchanged; it returns a new sampled DataFrame.
Supports an optional seed parameter for reproducible results.

PySpark sample() Function

The sample() function allows you to randomly select rows from a DataFrame. You can control whether to sample with replacement and define the fraction of rows to pick.

Syntax of sample()

Below is the syntax of the sample() function.


# Syntax of the sample() function
DataFrame.sample(withReplacement=False, fraction=0.1, seed=None)

Parameters

fraction: - Fraction of rows to generate, range [0.0, 1.0]. Note that it doesn't guarantee to provide the exact number of the fraction of records.
seed: - Seed for sampling (default a random seed). Used to reproduce the same random sampling.
withReplacement: - Sample with replacement or not (default False).

Return Value

Returns a new DataFrame containing randomly sampled rows based on the given fraction, leaving the original DataFrame unchanged.

Usage of sample() Function without Replacement

You can use the sample() function to select a subset of rows from a DataFrame randomly. First, create a DataFrame, then apply sample() to pick a fraction of rows without replacement.


# Sample 10% of the data without replacement
from pyspark.sql import SparkSession

# Create Spark session
spark = SparkSession.builder \
    .master("local[1]") \
    .appName("AIStudyLabs.com") \
    .getOrCreate()

# Create a DataFrame with 100 rows
dataset = spark.range(0, 100)
dataset.show()

# Randomly sample 10% of rows without replacement
sampled = dataset.sample(withReplacement=False, fraction=0.1, seed=42)
sampled.show()

Yields below the output.

Here, we randomly select about 10% of the rows from the dataset. The withReplacement=False ensures each row can only appear once in the sampled DataFrame. Using the seed parameter guarantees the same rows are selected each time we run the code.

Sample With Replacement

If you want some rows to appear more than once in the sampled dataset, you can set withReplacement=True.


# Get random samples with replacement
sample_df = df.sample(True, 0.1, seed=42)
print(sample_df.collect())

Yields below output.

Setting withReplacement=True allows Spark to pick the same row multiple times in the sampled result. This is useful when you want to simulate bootstrapping or repeated sampling scenarios.

If Fraction Greater Than 1

If you specify a fraction greater than 1 with withReplacement=True PySpark will select more rows than the original dataset because it allows repeated rows.


# Sample 150% of rows with replacement
sampled_large = dataset.sample(withReplacement=True, fraction=1.5, seed=1)
sampled_large.show(20)   # Show first 20 rows
print("Total rows:", sampled_large.count())

Yields below the output.


# Output:
+---+
| id|
+---+
|  5|
|  5|
| 18|
| 18|
| 21|
| 27|
| 27|
| 27|
| 39|
| 41|
| 41|
| 54|
| 61|
| 68|
| 73|
| 73|
| 85|
| 85|
| 93|
| 99|
+---+
only showing top 20 rows
Total rows: 147

Key Differences Between sample() and sampleBy()

The table below provides a complete comparison of the features of sample() and sampleBy().

| Feature | sample() | sampleBy() | | --- | --- | --- | | Type of Sampling | Simple random sampling | Stratified sampling | | Groups | Ignores groups | Considers groups (strata) | | Fraction | Applies to the entire DataFrame | Fraction can vary per group | | Replacement | Can be with or without replacement | Always without replacement | | Use Case | Random test/train splits, quick sampling | Balanced group-wise sampling |

Frequently Asked Questions of PySpark sample() Function

sample() → Performs simple random sampling across the entire DataFrame.
sampleBy() → Performs stratified sampling, letting you control the fraction for each group.

If withReplacement=True, the same row can appear multiple times in the sample (bootstrapping). If False, each row can appear only once.

A value between 0 and 1 picks that fraction of rows (e.g., 0.1 → 10%).
A value greater than 1 is allowed only when withReplacement=True and will return more rows than the original dataset.

The seed ensures reproducibility. Without it, each run may produce different sampled rows.

Conclusion

In this article, we explored PySpark’s sample() function for simple random sampling.

We learned how to sample rows with and without replacement.
We saw how to use the fraction and seed parameters.
We also compared sample() with sampleBy() to understand when to use each.

Use sample() when you need a quick, random subset of rows from your dataset, such as for testing, validation, or bootstrapping.

Happy Learning!!

Reference

https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.sample.html