From 61b550376c66c48619394fa5f3acad5738bf84bb Mon Sep 17 00:00:00 2001 From: brijrajk <22271048+brijrajk@users.noreply.github.com> Date: Fri, 12 Jun 2026 22:14:36 +0530 Subject: [PATCH] [SPARK-57418][DOCS] Add singleVariantColumn option to CSV, JSON, and XML data source options tables --- docs/sql-data-sources-csv.md | 6 ++++++ docs/sql-data-sources-json.md | 6 ++++++ docs/sql-data-sources-xml.md | 7 +++++++ 3 files changed, 19 insertions(+) diff --git a/docs/sql-data-sources-csv.md b/docs/sql-data-sources-csv.md index 8008bc562082c..a88b4ab149860 100644 --- a/docs/sql-data-sources-csv.md +++ b/docs/sql-data-sources-csv.md @@ -216,6 +216,12 @@ Data source options of CSV can be set via: Allows renaming the new field having malformed string created by PERMISSIVE mode. This overrides spark.sql.columnNameOfCorruptRecord. read + + singleVariantColumn + (none) + If specified, the entire CSV record is parsed and stored as a single column of VariantType with the given column name, instead of being split into individual fields. + read + multiLine false diff --git a/docs/sql-data-sources-json.md b/docs/sql-data-sources-json.md index e5c86054c09be..e9449ee17a4df 100644 --- a/docs/sql-data-sources-json.md +++ b/docs/sql-data-sources-json.md @@ -207,6 +207,12 @@ Data source options of JSON can be set via: Allows inferring of TimestampType and TimestampNTZType from strings that match the timestamp patterns defined by the timestampFormat and timestampNTZFormat options respectively. read + + singleVariantColumn + (none) + If specified, the entire JSON record is parsed and stored as a single column of VariantType with the given column name, instead of being split into individual fields. + read + enableDateTimeParsingFallback Enabled if the time parser policy has legacy settings or if no custom date or timestamp pattern was provided. diff --git a/docs/sql-data-sources-xml.md b/docs/sql-data-sources-xml.md index 949e6239e52ba..0dd2a0dde3be2 100644 --- a/docs/sql-data-sources-xml.md +++ b/docs/sql-data-sources-xml.md @@ -105,6 +105,13 @@ Data source options of XML can be set via: read + + singleVariantColumn + (none) + If specified, the entire XML record is parsed and stored as a single column of VariantType with the given column name, instead of being split into individual fields. + read + + attributePrefix _