Spark Timestamp - Extract hour, minute and second

Problem: How to extract or get an hour, minute and second from a Spark timestamp column?

Solution: Spark functions provides hour(), minute() and second() functions to extract hour, minute and second from Timestamp column respectively.

hour - function hour() extracts hour unit from Timestamp column or string column containing a timestamp.


Syntax : hour(e: Column): Column

minute - function minute() extracts minute unit from Timestamp column or string column containing a timestamp.


Syntax : minute(e: Column): Column

second - function second() extracts the second unit from the Timestamp column or string column containing a timestamp.


Syntax : second(e: Column): Column

Extract hour, minute and second example


package com.aistudylabs.spark.dataframe.functions.datetime

import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.functions.{col,hour,minute,second}

object GetTimeFromTimestamp extends App {

  val spark:SparkSession = SparkSession.builder()
    .master("local")
    .appName("AIStudyLabs.com")
    .getOrCreate()
  spark.sparkContext.setLogLevel("ERROR")

  import spark.sqlContext.implicits._

  val df = Seq(("2019-07-01 12:01:19.000"),
    ("2019-06-24 12:01:19.000"),
    ("2019-11-16 16:44:55.406"),
    ("2019-11-16 16:50:59.406")).toDF("input_timestamp")

  df.withColumn("hour", hour(col("input_timestamp")))
    .withColumn("minute", minute(col("input_timestamp")))
    .withColumn("second", second(col("input_timestamp")))
    .show(false)

}

Yields below output


+-----------------------+----+------+------+
|input_timestamp        |hour|minute|second|
+-----------------------+----+------+------+
|2019-07-01 12:01:19.000|12  |1     |19    |
|2019-06-24 12:01:19.000|12  |1     |19    |
|2019-11-16 16:44:55.406|16  |44    |55    |
|2019-11-16 16:50:59.406|16  |50    |59    |
+-----------------------+----+------+------+

The complete code can be downloaded from GitHub

Happy Learning !!

Spark Timestamp - Extract hour, minute and second

Extract hour, minute and second example

Related Articles