Built with Tutify.io

    AIStudyLabs

    Pandas DataFrame shift() Function

    Pandas DataFrame shift() Function

    In Pandas, the shift() function is used to shift the values in a DataFrame or Series along a particular axis. This is useful for creating lagged versions of data, which is often needed in time series analysis.

    In this article, I will explain the Pandas DataFrame shift() function by using its syntax, parameters, usage, and how to return a Series or DataFrame with data shifted by the designated number of periods.

    Key Points –

    <a id="h-pandas-dataframe-shift-introduction"></a>

    Pandas DataFrame shift() Introduction

    Let’s know the syntax of the Pandas DataFrame shift() function.

    
# Syntax of Pandas dataframe shift()
DataFrame.shift(periods=1, freq=None, axis=0, fill_value=None)

    <a id="h-parameters-of-the-dataframe-shift"></a>

    Parameters of the DataFrame shift()

    Following are the parameters of the DataFrame shift() function.

    <a id="h-return-value"></a>

    Return Value

    It returns a DataFrame or Series with the same shape and data type, but with the values shifted.

    <a id="h-usage-of-pandas-dataframe-shift-function"></a>

    Usage of Pandas DataFrame shift() Function

    The shift() function in Pandas is used to shift the values in a DataFrame or Series by a specified number of periods along a particular axis. It is particularly useful in time series analysis, data preprocessing, and creating lagged variables.

    Now, Let’s create Pandas DataFrame using data from a Python dictionary, where the columns are Column1, Column2.

    
# Create DataFrame 
import pandas as pd
import numpy as np

data = {'Column1': [2, 4, 6, 8, 10],
        'Column2': [3, 5, 7, 9, 11]}
df = pd.DataFrame(data)
print("Original DataFrame:\n",df)

    Yields below output.

    <a id="h-shifting-rows-downward-by-1-period-default"></a>

    Shifting Rows Downward by 1 Period (Default)

    To shift rows downward by 1 period using the default settings of the shift() function in Pandas, you can simply call the shift() method on your DataFrame or Series without any additional arguments.

    
# Shifting rows downward by 1 period
df2 = df.shift()
print("Shifted DataFrame:\n", df2)

    Here,

    <a id="h-shifting-rows-upward-by-2-periods"></a>

    Shifting Rows Upward by 2 Periods

    Alternatively, to shift rows upward by 2 periods in a DataFrame, you can use the shift() function with the periods parameter set to -2.

    
# Shifting rows upward by 2 periods
df2 = df.shift(periods=-2)
print("Shifted DataFrame:\n", df2)

# Output:
# Shifted DataFrame:
#     Column1  Column2
# 0      6.0      7.0
# 1      8.0      9.0
# 2     10.0     11.0
# 3      NaN      NaN
# 4      NaN      NaN

    Here,

    <a id="h-shifting-columns-to-the-right"></a>

    Shifting Columns to the Right

    To shift columns to the right in a DataFrame, you can use the shift() function with the axis parameter set to 1.

    
# Shifting columns to the right by 1 period
df2 = df.shift(periods=1, axis=1)
print("Shifted DataFrame:\n", df2)

# Output:
# Shifted DataFrame:
#     Column1  Column2
# 0      NaN      2.0
# 1      NaN      4.0
# 2      NaN      6.0
# 3      NaN      8.0
# 4      NaN     10.0

    Here,

    <a id="h-shifting-with-a-fill-value"></a>

    Shifting with a Fill Value

    When using the shift() function in Pandas, you can specify a fill_value parameter to replace missing values that are introduced as a result of shifting.

    
# Shifting rows downward by 1 period 
# With a fill value of 5
df2 = df.shift(fill_value=5)
print("Shifted DataFrame:\n", df2)

# Using a fill value
df2 = df.shift(periods=1, fill_value=5)
print("Shifted DataFrame:\n", df2)

# Output:
# Shifted DataFrame:
#     Column1  Column2
# 0        5        5
# 1        2        3
# 2        4        5
# 3        6        7
# 4        8        9

    Here,

    <a id="h-shifting-a-time-series-with-frequency"></a>

    Shifting a Time Series with Frequency

    Similarly, when working with time series data, the shift() function can be used with the freq parameter to shift the index by a specific frequency increment. This is particularly useful for aligning time series data or creating lagged features with a clear temporal interpretation.

    
import pandas as pd

# Creating a sample time series DataFrame
dates = pd.date_range('20220202', periods=5)
df_time = pd.DataFrame({'Value': [1, 2, 3, 4, 5]}, index=dates)

# Shifting the time series by 1 day
df2 = df_time.shift(periods=1, freq='D')
print("Shifting the time series:\n", df2)

# Output:
# Shifting the time series:
#              Value
# 2022-02-03      1
# 2022-02-04      2
# 2022-02-05      3
# 2022-02-06      4
# 2022-02-07      5

    Here,

    <a id="h-faq-on-pandas-dataframe-shift-function"></a>

    FAQ on Pandas DataFrame shift() Function

    The shift() function in Pandas shifts the values in a DataFrame or Series by a specified number of periods along a given axis. This can be useful for creating lagged versions of data, aligning data, and calculating differences between consecutive data points.

    To shift columns to the right by 1 period in a DataFrame, you can use the shift() function with the axis parameter set to 1.

    By default, shift() introduces NaN values in positions where data is shifted. You can replace these missing values with a specified scalar using the fill_value parameter.

    To shift rows downward by 1 period in a DataFrame, you can use the shift() function with its default parameters.

    You can use shift() with a MultiIndex DataFrame. The function will apply the shift to each level of the index based on the specified parameters.

    <a id="h-conclusion"></a>

    Conclusion

    In this article, you have explored the Pandas DataFrame shift() function, including its syntax, parameters, and usage. You also learned how to return a DataFrame or Series with the same shape and data type as the original, but with values shifted based on the specified parameters.

    Happy Learning!!

    <a id="h-related-articles"></a>

    Related Articles

    <a id="h-reference"></a>

    Reference