[SPARK-57030][SPARK-57031][DOCS] Document the TIME data type, functions, and ANSI behavior in the SQL reference

docs/sql-data-sources-avro.md

@@ -560,6 +560,11 @@ It also supports reading the following Avro [logical types](https://avro.apache.
     <td>long</td>
     <td>TimestampType</td>
   </tr>
+  <tr>
+    <td>time-micros</td>
+    <td>long</td>
+    <td>TimeType</td>
+  </tr>
   <tr>
     <td>decimal</td>
     <td>fixed</td>
@@ -603,6 +608,11 @@ Spark supports writing of all Spark SQL types into Avro. For most types, the map
     <td>long</td>
     <td>timestamp-micros</td>
   </tr>
+  <tr>
+    <td>TimeType</td>
+    <td>long</td>
+    <td>time-micros</td>
+  </tr>
   <tr>
     <td>DecimalType</td>
     <td>fixed</td>

docs/sql-data-sources-csv.md

@@ -180,6 +180,12 @@ Data source options of CSV can be set via:
     <td>Sets the string that indicates a timestamp without timezone format. Custom date formats follow the formats at <a href="https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html">Datetime Patterns</a>. This applies to timestamp without timezone type, note that zone-offset and time-zone components are not supported when writing or reading this data type.</td>
     <td>read/write</td>
   </tr>
+  <tr>
+    <td><code>timeFormat</code></td>
+    <td>HH:mm:ss</td>
+    <td>Sets the string that indicates a time format. Custom time formats follow the formats at <a href="https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html">Datetime Patterns</a>. This applies to time type.</td>
+    <td>read/write</td>
+  </tr>
   <tr>
     <td><code>enableDateTimeParsingFallback</code></td>
     <td>Enabled if the time parser policy has legacy settings or if no custom date or timestamp pattern was provided.</td>

docs/sql-data-sources-json.md

@@ -201,10 +201,16 @@ Data source options of JSON can be set via:
     <td>Sets the string that indicates a timestamp without timezone format. Custom date formats follow the formats at <a href="https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html">Datetime Patterns</a>. This applies to timestamp without timezone type, note that zone-offset and time-zone components are not supported when writing or reading this data type.</td>
     <td>read/write</td>
   </tr>
+  <tr>
+    <td><code>timeFormat</code></td>
+    <td>HH:mm:ss</td>
+    <td>Sets the string that indicates a time format. Custom time formats follow the formats at <a href="https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html">Datetime Patterns</a>. This applies to time type.</td>
+    <td>read/write</td>
+  </tr>
   <tr>
     <td><code>inferTimestamp</code></td>
     <td><code>false</code></td>
-    <td>Allows inferring of <code>TimestampType</code> and <code>TimestampNTZType</code> from strings that match the timestamp patterns defined by the <code>timestampFormat</code> and <code>timestampNTZFormat</code> options respectively.</td>
+    <td>Allows inferring of <code>TimestampType</code> and <code>TimestampNTZType</code> from strings that match the timestamp patterns defined by the <code>timestampFormat</code> and <code>timestampNTZFormat</code> options respectively. When enabled, it also allows inferring of <code>TimeType</code> from strings that match the time pattern defined by the <code>timeFormat</code> option.</td>
     <td>read</td>
   </tr>
   <tr>

docs/sql-data-sources-xml.md

@@ -188,6 +188,13 @@ Data source options of XML can be set via:
     <td>read/write</td>
   </tr>
 
+  <tr>
+    <td><code>timeFormat</code></td>
+    <td><code>HH:mm:ss</code></td>
+    <td>Sets the string that indicates a time format. Custom time formats follow the formats at <a href="https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html"> datetime pattern</a>. This applies to time type.</td>
+    <td>read/write</td>
+  </tr>
+
   <tr>
     <td><code>locale</code></td>
     <td><code>en-US</code></td>

docs/sql-ref-ansi-compliance.md

@@ -257,6 +257,8 @@ At the heart of this conflict resolution is the Type Precedence List which defin
 
 \*\*\* For a complex type, the precedence rule applies recursively to its component elements.
 
+The `TIME` type does not promote to any other type. Note that Spark's `TIME` type deviates from the SQL standard in two ways: the default fractional-seconds precision is `6` (the ANSI default is `0`), and `TIME WITH TIME ZONE` is not supported.
+
 Special rules apply for untyped NULL. A NULL can be promoted to any other type.
 
 This is a graphical depiction of the precedence list as a directed tree:

docs/sql-ref-datatypes.md

@@ -47,7 +47,7 @@ Spark SQL and DataFrames support the following data types:
   - `DateType`: Represents values comprising values of fields year, month and day, without a
   time-zone.
   - `TimeType(precision)`: Represents values comprising values of fields hour, minute and second with the number of decimal digits `precision` following the decimal point in the seconds field, without a time-zone.
-  The range of values is from `00:00:00` to `23:59:59` for min precision `0`, and to `23:59:59.999999` for max precision `6`.
+  The range of values is from `00:00:00` to `23:59:59` for min precision `0`, and to `23:59:59.999999999` for max precision `9`. The default precision is `6`.
   - `TimestampType`: Timestamp with local time zone(TIMESTAMP_LTZ). It represents values comprising values of fields year, month, day,
   hour, minute, and second, with the session local time-zone. The timestamp value represents an
   absolute point in time.
@@ -143,6 +143,7 @@ from pyspark.sql.types import *
 |**TimestampType**|datetime.datetime|TimestampType()|
 |**TimestampNTZType**|datetime.datetime|TimestampNTZType()|
 |**DateType**|datetime.date|DateType()|
+|**TimeType**|datetime.time|TimeType()|
 |**DayTimeIntervalType**|datetime.timedelta|DayTimeIntervalType()|
 |**GeometryType**|Geometry|GeometryType(*srid*)<br/>**Note:** *srid* is required and may be an `int` or the string `"ANY"`.|
 |**GeographyType**|Geography|GeographyType(*srid*)<br/>**Note:** *srid* is required and may be an `int` or the string `"ANY"`.|
@@ -241,6 +242,7 @@ please use factory methods provided in
 |**BooleanType**|logical|"bool"|
 |**TimestampType**|POSIXct|"timestamp"|
 |**DateType**|Date|"date"|
+|**TimeType**|Not supported|Not supported|
 |**GeometryType**|Not supported|Not supported|
 |**GeographyType**|Not supported|Not supported|
 |**ArrayType**|vector or list|list(type="array", elementType=*elementType*, containsNull=[*containsNull*])<br/>**Note:** The default value of *containsNull* is TRUE.|
@@ -264,6 +266,7 @@ The following table shows the type names as well as aliases used in Spark SQL pa
 |**FloatType**|FLOAT, REAL|
 |**DoubleType**|DOUBLE|
 |**DateType**|DATE|
+|**TimeType**|TIME, TIME(p)|
 |**TimestampType**|TIMESTAMP, TIMESTAMP_LTZ|
 |**TimestampNTZType**|TIMESTAMP_NTZ|
 |**StringType**|STRING|

docs/sql-ref-datetime-pattern.md

@@ -23,10 +23,10 @@ There are several common scenarios for datetime usage in Spark:
 
 - CSV/JSON datasources use the pattern string for parsing and formatting datetime content.
 
-- Datetime functions related to convert `StringType` to/from `DateType` or `TimestampType`.
-  For example, `unix_timestamp`, `date_format`, `to_unix_timestamp`, `from_unixtime`, `to_date`, `to_timestamp`, `from_utc_timestamp`, `to_utc_timestamp`, etc.
+- Datetime functions related to convert `StringType` to/from `DateType`, `TimeType` or `TimestampType`.
+  For example, `unix_timestamp`, `date_format`, `to_unix_timestamp`, `from_unixtime`, `to_date`, `to_time`, `to_timestamp`, `from_utc_timestamp`, `to_utc_timestamp`, etc.
 
-Spark uses pattern letters in the following table for date and timestamp parsing and formatting:
+Spark uses pattern letters in the following table for date, time and timestamp parsing and formatting:
 
 |Symbol|Meaning|Presentation|Examples|
 |------|-------|------------|--------|

docs/sql-ref-functions-builtin.md

@@ -51,7 +51,7 @@ user-defined function that may share the same name.
 #### Examples
 {% include_api_gen generated-map-funcs-examples.html %}
 
-### Date and Timestamp Functions
+### Date, Time and Timestamp Functions
 {% include_api_gen generated-datetime-funcs-table.html %}
 #### Examples
 {% include_api_gen generated-datetime-funcs-examples.html %}

docs/sql-ref-functions.md

@@ -24,15 +24,15 @@ Built-in functions are commonly used routines that Spark SQL predefines and a co
 
 ### Built-in Functions
 
-Spark SQL has some categories of frequently-used built-in functions for aggregation, arrays/maps, date/timestamp, and JSON data.
+Spark SQL has some categories of frequently-used built-in functions for aggregation, arrays/maps, date/time/timestamp, and JSON data.
 This subsection presents the usages and descriptions of these functions.
 
 #### Scalar Functions
  * [Array Functions](sql-ref-functions-builtin.html#array-functions)
  * [Collection Functions](sql-ref-functions-builtin.html#collection-functions)
  * [Struct Functions](sql-ref-functions-builtin.html#struct-functions)
  * [Map Functions](sql-ref-functions-builtin.html#map-functions)
- * [Date and Timestamp Functions](sql-ref-functions-builtin.html#date-and-timestamp-functions)
+ * [Date, Time and Timestamp Functions](sql-ref-functions-builtin.html#date-time-and-timestamp-functions)
  * [Mathematical Functions](sql-ref-functions-builtin.html#mathematical-functions)
  * [String Functions](sql-ref-functions-builtin.html#string-functions)
  * [Bitwise Functions](sql-ref-functions-builtin.html#bitwise-functions)

docs/sql-ref-literals.md

@@ -433,7 +433,7 @@ SELECT DATE '2011-11-11' AS col;
 ```sql
 TIME { '[h]h:[m]m[:]' |
        '[h]h:[m]m:[s]s[.]' |
-       '[h]h:[m]m:[s]s.[ms][ms][ms][us][us][us]'}
+       '[h]h:[m]m:[s]s.[ms][ms][ms][us][us][us][ns][ns][ns]'}
 ```
 **Note:** defaults to `00` if hour, minute or second is not specified.
 
@@ -464,6 +464,12 @@ SELECT TIME'23:59:59.999999' as col;
 +---------------+
 |23:59:59.999999|
 +---------------+
+SELECT TIME'23:59:59.999999999' as col;
++------------------+
+|col               |
++------------------+
+|23:59:59.999999999|
++------------------+
 ```
 
 #### Timestamp Syntax

docs/sql-ref-syntax-aux-describe-table.md

@@ -154,6 +154,7 @@ to return the metadata pertaining to a partition or column respectively.
 | BinaryType            | `{ "name" : "binary" }`                                                                                                                                          |
 | BooleanType           | `{ "name" : "boolean" }`                                                                                                                                         |
 | DateType              | `{ "name" : "date" }`                                                                                                                                            |
+| TimeType              | `{ "name" : "time(p)" }`                                                                                                                                         |
 | VariantType           | `{ "name" : "variant" }`                                                                                                                                         |
 | TimestampType         | `{ "name" : "timestamp_ltz" }`                                                                                                                                   |
 | TimestampNTZType      | `{ "name" : "timestamp_ntz" }`                                                                                                                                   |

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala

@@ -1209,14 +1209,16 @@ case class DayName(child: Expression) extends GetDateField with DefaultStringPro
   usage = "_FUNC_(timestamp, fmt) - Converts `timestamp` to a value of string in the format specified by the date format `fmt`.",
   arguments = """
     Arguments:
-      * timestamp - A date/timestamp or string to be converted to the given format.
+      * timestamp - A date, time, timestamp or string to be converted to the given format.
       * fmt - Date/time format pattern to follow. See <a href="https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html">Datetime Patterns</a> for valid date
               and time format patterns.
   """,
   examples = """
     Examples:
       > SELECT _FUNC_('2016-04-08', 'y');
        2016
+      > SELECT _FUNC_(TIME'14:30:45', 'HH:mm:ss');
+       14:30:45
   """,
   group = "datetime_funcs",
   since = "1.5.0")
@@ -3874,11 +3876,11 @@ object DatePart {
 
 // scalastyle:off line.size.limit
 @ExpressionDescription(
-  usage = "_FUNC_(field, source) - Extracts a part of the date/timestamp or interval source.",
+  usage = "_FUNC_(field, source) - Extracts a part of the date, time, timestamp, or interval source.",
   arguments = """
     Arguments:
       * field - selects which part of the source should be extracted, and supported string values are as same as the fields of the equivalent function `EXTRACT`.
-      * source - a date/timestamp or interval column from where `field` should be extracted
+      * source - a date, time, timestamp, or interval column from where `field` should be extracted
   """,
   examples = """
     Examples:
@@ -3898,6 +3900,8 @@ object DatePart {
        11
       > SELECT _FUNC_('MINUTE', INTERVAL '123 23:55:59.002001' DAY TO SECOND);
        55
+      > SELECT _FUNC_('HOUR', TIME'09:08:01.000001');
+       9
   """,
   note = """
     The _FUNC_ function is equivalent to the SQL-standard function `EXTRACT(field FROM source)`