How to Generate Random Number between 1 and 100 in PySpark?

How to generate a random number between 1 and 100 in PySpark? You can use PySpark built-in functions like rand(), randn(), and custom approaches such as using arrays to generate both integers and decimals. By combining these functions with transformations like floor(), you can easily generate random integers between 1 and 100 or random decimal values for simulations.

In PySpark, generating random numbers is useful for tasks such as creating synthetic datasets, testing, sampling, and data simulation.

In this article, I will explain how to generate random numbers between 1 and 100 in PySpark using different functions and approaches.

Key Points-

• rand() Function → Generates uniform random numbers between 0.0 and 1.0.
• Scaling Technique → Multiply by 100 and add 1 to shift values to the range 1–100.
• Casting to Integer → Use floor() or cast("int") to convert decimals into integers.
• rand(seed) → Adding a seed ensures reproducibility of random numbers.
• randn() Function → Generates random numbers from a standard normal distribution (mean = 0, std = 1).
• Scaling randn() → You can normalize and scale randn() output to fit into the 1–100 range.
• Decimal Random Numbers → Keeping the raw rand() output gives random decimals between 0 and 1.
• Array Choice → By using sequence() and a custom array_choice logic, you can directly pick random numbers from 1 to 100.
• Non-Destructive → These operations add new columns to the DataFrame, keeping the original data intact.
• Practical Uses → Useful for test/train splits, generating synthetic IDs, simulations, and randomized sampling.

<a id="h-pyspark-rand"></a>

PySpark rand

In PySpark, the rand() function is used to generate random numbers between 0 and 1. It is part of the pyspark.sql.functions module and is widely used for sampling, random splits, assigning random values to DataFrames, or generating random seeds for reproducibility.

Syntax

# Syntax of the rand()
randn(seed=None)

Parameters

• seed: (optional, int) A seed value to ensure reproducible random numbers. If not provided, results vary each run.

Return Value

• Returns a column of random numbers uniformly distributed between 0.0 and 1.0.

<a id="h-using-rand-to-generate-random-integers-between-1-and-100"></a>

Using rand() to Generate Random Integers Between 1 and 100

You can use rand() to generate random decimal values between 0 and 1, then scale them by multiplying with 100 and adding 1. Finally, applying floor() converts these values into random integers between 1 and 100.

# Generate Random Integers Between 1 and 100
from pyspark.sql import SparkSession
from pyspark.sql.functions import rand, floor

# Create Spark session
spark = SparkSession.builder \
    .master("local[1]") \
    .appName("RandomNumberExample") \
    .getOrCreate()

# Generate DataFrame with random numbers
df = spark.range(10).withColumn("random_num", floor(rand() * 100 + 1))

df.show()

Yields below the output.

<a id="h-use-a-seed-to-get-repeated-random-numbers-between-1-and-100"></a>

Use a seed to Get Repeated Random Numbers between 1 and 100

You can also generate reproducible random numbers in PySpark by setting a seed in the rand() function. In this example, each run will return the same random integers between 1 and 100.

# Generate reproducable random numbers between 1 and 100
df = spark.range(5).withColumn("random_num", floor(rand(seed=42) * 100 + 1))
df.show()

Yields below the output.

<a id="h-create-new-column-with-random-decimal-numbers"></a>

Create New Column with Random Decimal Numbers

Instead of integers, you can directly use rand() to generate decimal numbers between 0 and 1.

# Create New Column with Random Decimal Numbers
from pyspark.sql import SparkSession
from pyspark.sql.functions import rand

# Create Spark session
spark = SparkSession.builder \
    .master("local[1]") \
    .appName("RandomDecimalExample") \
    .getOrCreate()

# Create DataFrame with 5 rows
df = spark.range(5).withColumnRenamed("id", "row_id")

# Add a new column with random decimal numbers (0.0 to 1.0)
df = df.withColumn("random_decimal", rand())

df.show(truncate=False)

Yields below output.

# Output:
+------+-------------------+
|row_id|random_decimal     |
+------+-------------------+
|0     |0.01167529305023951|
|1     |0.14559014241473667|
|2     |0.7402370797592487 |
|3     |0.12049598325376076|
|4     |0.702652275541713  |
+------+-------------------+

<a id="h-using-randn-to-generate-normal-distribution-values"></a>

Using randn() to Generate Normal Distribution Values

The randn() function generates random values from a standard normal distribution (mean = 0, variance = 1). You can scale these values to fit within a desired range, such as 1 to 100.

Syntax

# Syntax of the randn()
randn(seed=None)

Parameters

• seed: (optional, int) A seed value for reproducibility.

Return Value

• Returns a Column of random numbers following a standard normal distribution.

This code generates random values from a standard normal distribution using randn() and then scales them into the range 1 to 100. Finally, it creates a new column random_num containing these scaled random integers.

from pyspark.sql.functions import randn, col, min, max

# Generate random normal values
df = spark.range(10).withColumn("randn_val", randn(seed=1))

# Scale to range 1-100 using transformation
scaled_df = df.withColumn("random_num", floor((col("randn_val") - (-3)) / (3 - (-3)) * 99 + 1))

scaled_df.select("random_num").show()

Yields below output.

# Output:
+----------+
|random_num|
+----------+
|        78|
|        70|
|        62|
|        57|
|        43|
|        41|
|        24|
|        74|
|        53|
|        31|
+----------+

<a id="h-use-custom-array-choice-for-random-numbers"></a>

Use Custom array_choice() for Random Numbers

You can also define a custom function to pick random values from an array.
This approach is helpful when you want to choose random elements from a predefined range.

# Define an array_choice function for random numbers
from pyspark.sql import SparkSession
from pyspark.sql.functions import col, rand, size, sequence, lit

# Define array_choice function
def array_choice(col_array):
    # Pick a random index from the array
    idx = (rand() * size(col_array)).cast("int")
    return col_array[idx]

# Create Spark session
spark = SparkSession.builder \
    .master("local[1]") \
    .appName("ArrayChoiceRandomExample") \
    .getOrCreate()

# Create DataFrame with 5 rows
df = spark.range(5).withColumnRenamed("id", "row_id")

# Create array column [1, 2, ..., 100]
df = df.withColumn("numbers_array", sequence(lit(1), lit(100)))

# Pick a random number from the array
df = df.withColumn("random_number", array_choice(col("numbers_array")))

df.select("row_id", "random_number").show(truncate=False)

Yields below output.

# Output:
+------+-------------+
|row_id|random_number|
+------+-------------+
|0     |73           |
|1     |72           |
|2     |46           |
|3     |97           |
|4     |16           |
+------+-------------+

<a id="h-rand-vs-randn"></a>

rand() vs randn()

The table below highlights the differences between the rand() and randn() functions.

| Function | Distribution | Range (Default) | Use Case | | --- | --- | --- | --- | | rand() | Uniform | 0 to 1 | Generating evenly distributed random numbers | | randn() | Normal | -∞ to ∞ (centered at 0) | Generating normally distributed random numbers |

<a id="h-frequently-asked-questions-of-generate-random-number-between-1-and-100-in-pyspark"></a>

Frequently Asked Questions of Generate Random Number between 1 and 100 in PySpark

rand() generates values uniformly between 0 and 1.

Multiply rand() by 100, add 1, and apply floor() to get integers in the 1–100 range.

The seed ensures reproducibility — running the code multiple times with the same seed gives identical random numbers.

Simply use rand() without applying floor(). This gives values between 0.0 and 1.0 (or scale further if needed).

Use rand() for uniform distribution.
Use randn() for normal distribution.
Apply transformations to adjust the ranges as required.

<a id="h-conclusion"></a>

Conclusion

In this article, I explained how to generate random numbers between 1 and 100 in PySpark.

We used rand() to generate uniform random values and scaled them to integers between 1 and 100.
We explored rand(seed) for reproducible results.
We discussed randn() for normal distribution values and how to scale them into the desired range.
We also introduced a custom array_choice approach to directly pick random numbers from a defined array of values.
Finally, we looked at practical examples for generating synthetic data.

Use these methods whenever you need random data generation in PySpark — whether for sampling, simulations, or creating test datasets.

Happy Learning!!

<a id="h-related-articles"></a>

Related Articles