pandas.DataFrame.fillna() - Explained by Examples

pandas.DataFrame.fillna() method is used to fill column (one or multiple columns) containing NA/NaN/None with 0, empty, blank, or any specified values etc. NaN is considered a missing value. When you dealing with machine learning, handling missing values is very important, not handling these will result in a side effect with an incorrect result.

CSV files are received from third-party sources, most of the time it has null values for blanks/empty. By using pandas.read_csv() we can load the CSV file into DataFrame and pandas converts all null values into NaN in DataFrame.

Either you can drop rows with NaN values using pandas.DataFrame.dropna() or handle NaN by filling with specific values using the fillna() method.

Key Points –

fillna() is used to replace NaN (missing) values in a DataFrame with specified values or methods.
You can provide a scalar, dictionary, or Series to the value parameter to replace missing data.
It is used to fill NaN values with specified values (0, blank, e.t.c).
If you want to consider infinity (inf and -inf) to be “NA” in computations, you can set pandas.options.mode.use_inf_as_na = True.
Besides NaN, pandas None also considered as missing.
You can target specific columns to fill by calling fillna() on a subset of the DataFrame or passing a dictionary mapping columns to fill values.
fillna() can be chained with other DataFrame methods for streamlined data cleaning workflows.

Quick Examples of Pandas fillna()

Below are bow quick examples and usage of the pandas fillna() method.


# Quick examples of pandas fillna()

# Example 1: Fillna() on all columns
df2=df.fillna('None')

# Example 2: Fillna() on once column
df2['Discount'] =  df['Discount'].fillna(0)

# Example 3: Fillna() on multiple columns
df2[['Discount','Fee']] =  df[['Discount','Fee']].fillna(0)

# Example 4: Fillna() on multiple columns with different values
df2 =  df.fillna(value={'Discount':0,'Fee':10000})

# Example 5: Fill with limit
df2=df.fillna(value={'Discount':0,'Fee':0},limit=1)

Syntax of Pandas DataFrame fillna()

Below is the syntax of pandas.DataFrame.fillna() method. This takes parameters value, method, axis, inplace, limit, and downcast and returns a new DataFrame. When inplace=True is used, it returns None as the replace happens on the existing DataFrame object.


# Syntax of pandas.DataFrame.fillna()
DataFrame.fillna(value=None, method=None, axis=None, inplace=False, limit=None, downcast=None)

Parameters of the DataFrame fillna()

Following are the parameters of the fillna() function.

value - Takes either scalar, dict, Series, or DataFrame but not list.
method - Takes one of these values {‘backfill’, ‘bfill’, ‘pad’, ‘ffill’, None}. Default None.
axis - 0 or ‘index’, 1 or ‘columns’. Used to specify the axis to fill the values.
inplace - Default False. When used True, it updates the existing DataFrame object.
limit - Specify how many fills should happen. This is the maximum number of consecutive NaN values replaced with specified values.
downcast - It takes a dict of key-value pair that specifies data type to downcast. Like Float64 to int64, date to string e.t.c

Let's create a DataFrame


# Create DataFrame
import pandas as pd
import numpy as np
df = pd.DataFrame(({
     'Courses':["Spark",'Java',"Scala",'Python'],
     'Fee' :[20000,np.nan,26000,24000],
     'Duration':['30days','40days', pd.NA,'40days'],
     'Discount':[1000,np.nan,2500,None]
               }))
print("Create DataFrame:\n", df)

Yields below output.

Pandas Fillna NaN with None Value

fillna() method is used to fill NaN/NA values on a specified column or on an entire DataaFrame with any given value. You can specify modifications using inplace, or limit how many fillings to perform, or choose an axis whether to fill on rows/columns etc. The Below example fills all NaN values with the None value.


# Fillna to replace all NaN
df2 = df.fillna('None')
print("After replacing all NAN/NA values with None:\n", df2)

Yields below output.

In order to update the existing DataFrame use df.fillna('None', inplace=True). You can also use pandas.DataFrame.replace() method to replace NaN with 0 value. similarly, you can also replace NaN with blank or empty string.

Pandas Fillna on One Column

The above example filled all NaN values on the entire DataFrame. Sometimes you would need to replace just on one column, you can do so by selecting the DataFrame column for the fillna() method.


# Fillna on one column
df2['Discount'] =  df['Discount'].fillna('0')
print(df2)

# Outputs:
#  Courses      Fee Duration Discount
# 0   Spark  20000.0   30days   1000.0
# 1    Java     None   40days        0
# 2   Scala  26000.0     None   2500.0
# 3  Python  24000.0   40days        0

Fillna on Multiple Columns

Use the pandas fillna() method to fill a specified value on multiple DataFrame columns, the below example updates columns Discount and Fee with 0 for NaN values.


# Fillna() on multiple columns
df2[['Discount','Fee']] =  df[['Discount','Fee']].fillna('0')
print(df2)

# Outputs:
  Courses      Fee Duration Discount
0   Spark  20000.0   30days   1000.0
1    Java        0   40days        0
2   Scala  26000.0     None   2500.0
3  Python  24000.0   40days        0

Now, let's see how to fill different values for each column. The below example updates a column Discount with 0 and column Fee with 10000 for NaN values.


# Fillna() on multiple columns
df2 =  df.fillna(value={'Discount':'0','Fee':10000})
print(df2)

# Outputs:
#  Courses      Fee Duration Discount
# 0   Spark  20000.0   30days   1000.0
# 1    Java  10000.0   40days        0
# 2   Scala  26000.0      NaN   2500.0
# 3  Python  24000.0   40days        0

Fill with Limit Param

To control how to NaN values to fill use the limit param. Compare the below result with the above one to see the differences.


# Fill with limit
df2=df.fillna(value={'Discount':0,'Fee':0},limit=1)
print(df2)

# Outputs:
#  Courses      Fee Duration  Discount
# 0   Spark  20000.0   30days    1000.0
# 1    Java      0.0   40days       0.0
# 2   Scala  26000.0      NaN    2500.0
# 3  Python  24000.0   40days       NaN

Complete Example of Pandas Fillna


import pandas as pd
import numpy as np
df = pd.DataFrame(({
     'Courses':["Spark",'Java',"Scala",'Python'],
     'Fee' :[20000,np.nan,26000,24000],
     'Duration':['30days','40days',np.nan,'40days'],
     'Discount':[1000,np.nan,2500,None]
               }))
print(df)

# Fillna() on all columns
df2=df.fillna('None')
print(df2)

# Fillna() on once column
df2['Discount'] =  df['Discount'].fillna(0)
print(df2)

# Fillna() on multiple columns
df2[['Discount','Fee']] =  df[['Discount','Fee']].fillna(0)
print(df2)

# Fillna() on multiple columns
df2 =  df.fillna(value={'Discount':0,'Fee':10000})
print(df2)

# Fill with limit
df2=df.fillna(value={'Discount':0,'Fee':0},limit=1)
print(df2)

FAQ on Pandas DataFrame.fillna()

fillna() is used to replace missing values (NaN) in a DataFrame or Series with a specified value or method (like forward/backward fill).

You can replace NaN (missing values) with a specific value in a Pandas DataFrame or Series using the .fillna() method.

To fill NaN values with different values for different columns, you can use a dictionary with the .fillna() method. The dictionary specifies the replacement value for each column.

You can limit the number of NaN values to fill using the limit parameter in the .fillna() method. This allows you to specify how many NaN values should be replaced in each column or the entire DataFrame.

By default, the .fillna() method does not modify the original DataFrame. It returns a new DataFrame with the missing values filled. If you want to modify the original DataFrame in place, you can set the inplace parameter to True.

Conclusion

In this article, you have learned the DataFrame fillna() method to fill one column, or multiple columns containing NaN with a specified value. Also learned to replace different values for each column.

Happy Learning !!