Introduction

Following are sort functions

| SPARK SQL SORT FUNCTION SYNTAX | SPARK FUNCTION DESCRIPTION | | ---------------------------------------------- | ------------------------------------------------------------------------------------------------- | | asc(columnName: String): Column | asc function is used to specify the ascending order of the sorting column on DataFrame or DataSet | | asc_nulls_first(columnName: String): Column | Similar to asc function but null values return first and then non-null values | | asc_nulls_last(columnName: String): Column | Similar to asc function but non-null values return first and then null values | | desc(columnName: String): Column | desc function is used to specify the descending order of the DataFrame or DataSet sorting column. | | desc_nulls_first(columnName: String): Column | Similar to desc function but null values return first and then non-null values. | | desc_nulls_last(columnName: String): Column | Similar to desc function but non-null values return first and then null values. |

asc() – ascending function

asc function is used to specify the ascending order of the sorting column on DataFrame or DataSet.

Example in Scala:

import org.apache.spark.sql.functions.{asc}
val df = Seq((1, "apple"), (3, "banana"), (2, "lemon")).toDF("id", "fruit")
df.sort(asc("id")).show()

output:

+---+-------+
| id|  fruit|
+---+-------+
|  1|  apple|
|  2|  lemon|
|  3| banana|
+---+-------+

asc_nulls_first() – ascending with nulls first

Similar to asc function but null values return first and then non-null values.

Example in Scala:

import org.apache.spark.sql.functions.{asc_nulls_first}
val df = Seq((null, "kiwi"), (3, "banana"), (2, "lemon")).toDF("id", "fruit")
df.sort(asc_nulls_first("id")).show()

output:

+----+-------+
|  id|  fruit|
+----+-------+
|null|   kiwi|
|   2|  lemon|
|   3| banana|
+----+-------+

asc_nulls_last()  – ascending with nulls last

Similar to asc function but non-null values return first and then null values.

Example in Scala:

import org.apache.spark.sql.functions.{asc_nulls_last}
val df = Seq((null, "kiwi"), (3, "banana"), (2, "lemon")).toDF("id", "fruit")
df.sort(asc_nulls_last("id")).show()

output:

+----+-------+
|  id|  fruit|
+----+-------+
|   2|  lemon|
|   3| banana|
|null|   kiwi|
+----+-------+

desc() – descending function

desc function is used to specify the descending order of the DataFrame or DataSet sorting column.

Example in Scala:

import org.apache.spark.sql.functions.{desc}
val df = Seq((1, "apple"), (3, "banana"), (2, "lemon")).toDF("id", "fruit")
df.sort(desc("id")).show()

output:

+---+-------+
| id|  fruit|
+---+-------+
|  3| banana|
|  2|  lemon|
|  1|  apple|
+---+-------+

desc_nulls_first() – descending with nulls first

Similar to desc function but null values return first and then non-null values.

Example in Scala:

import org.apache.spark.sql.functions.{desc_nulls_first}
val df = Seq((null, "kiwi"), (3, "banana"), (2, "lemon")).toDF("id", "fruit")
df.sort(desc_nulls_first("id")).show()

output:

+----+-------+
|  id|  fruit|
+----+-------+
|null|   kiwi|
|   3| banana|
|   2|  lemon|
+----+-------+

desc_nulls_last() – descending with nulls last

Similar to desc function but non-null values return first and then null values.

Example in Scala:

import org.apache.spark.sql.functions.{desc_nulls_last}
val df = Seq((null, "kiwi"), (3, "banana"), (2, "lemon")).toDF("id", "fruit")
df.sort(desc_nulls_last("id")).show()

output:

+----+-------+
|  id|  fruit|
+----+-------+
|   3| banana|
|   2|  lemon|
|null|   kiwi|
+----+-------+