Built with Tutify.io

    AIStudyLabs

    Spark date_format() - Convert Timestamp to String

    Spark date_format() - Convert Timestamp to String

    Problem: How to convert the Spark Timestamp column to String on DataFrame column?

    Solution: Using <em>date_format</em>() Spark SQL date function, we can convert Timestamp to the String format. Spark support all Java Data formatted patterns for conversion. In this article, we will see a few examples in the Scala language.

    <a id="h-1-complete-example-of-converting-timestamp-to-string"></a>

    1. Complete example of converting Timestamp to String

    In this example, I am using Spark current_timestamp() to get the current system timestamp and then we convert this to different string patterns.

    
package com.aistudylabs.spark.dataframe.functions.datetime

import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.functions.{current_date, current_timestamp, date_format}

object TimestampToString extends App {

  val spark:SparkSession = SparkSession.builder()
    .master("local")
    .appName("AIStudyLabs.com")
    .getOrCreate()
  spark.sparkContext.setLogLevel("ERROR")

  import spark.sqlContext.implicits._

  Seq(1).toDF("seq").select(
    current_timestamp().as("current_date"),
    date_format(current_timestamp(),"yyyy MM dd").as("yyyy MM dd"),
    date_format(current_timestamp(),"MM/dd/yyyy hh:mm").as("MM/dd/yyyy"),
    date_format(current_timestamp(),"yyyy MMM dd").as("yyyy MMMM dd"),
    date_format(current_timestamp(),"yyyy MMMM dd E").as("yyyy MMMM dd E")
  ).show(false)

}

    Yields below output:

    
// Output:
+------------+----------+----------------+------------+--------------------+
|current_date|yyyy MM dd|MM/dd/yyyy      |yyyy MMMM dd|yyyy MMMM dd E      |
+------------+----------+----------------+------------+--------------------+
|2019-11-17  |2019 11 17|11/17/2019 02:44|2019 Nov 17 |2019 November 17 Sun|
+------------+----------+----------------+------------+--------------------+

    Happy Learning !!

    <a id="h-related-articles"></a>

    Related Articles