test: add ALTER TABLE tests validating existing column mapping annotations by sanujbasu · Pull Request #2502 · delta-io/delta-kernel-rs

sanujbasu · 2026-04-30T05:31:37Z

What changes are proposed in this pull request?

ALTER TABLE ADD COLUMN flows through the same column-mapping preserve/fill/assign
path as CREATE (try_assign_field_column_mapping -> apply_schema_operations). These
integration tests pin that contract end-to-end through the alter -> snapshot-reload
roundtrip:

id + physicalName preserved when fully supplied (name and id modes)
id allocated when only physicalName is supplied
physicalName filled when only id is supplied
supplied id below the existing maxColumnId is accepted when it does not collide
(matches delta-spark; diverges from the Java Kernel proposal in [Kernel] Update column mapping and schema evolution code to support usage with replace table delta#4520)
supplied id that collides with an existing field is rejected

Refs #2377.

How was this change tested?

New integration tests in kernel/tests/integration/features/alter_table.rs, run with
cargo test -p delta_kernel --all-features --test integration. fmt, clippy, and the
full suite pass.

codecov · 2026-04-30T05:36:16Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.64%. Comparing base (b7b23e6) to head (2e1694f).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2502   +/-   ##
=======================================
  Coverage   88.64%   88.64%           
=======================================
  Files         216      216           
  Lines       69450    69450           
  Branches    69450    69450           
=======================================
  Hits        61567    61567           
  Misses       5576     5576           
  Partials     2307     2307

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

github-actions · 2026-06-19T01:29:17Z

Benchmark results: ✅ Pass

Summary: 🚀 0 · ✅ 10 · ☑️ 3 · 🚧 1 · ❌ 0

Per-benchmark results (14 rows)

Test	Change	Base	PR
`clustered/readMetadataLatestPredicate/serial`	✅ 1.01x faster	104.8±1.26ms	104.0±1.41ms
`crcLatest/snapshotLatest`	✅ 1.01x faster	11.6±0.29ms	11.5±0.34ms
`crcMissing/snapshotLatest`	✅ 1.01x faster	27.7±0.59ms	27.5±0.72ms
`crcSlightlyStale/snapshotLatest`	☑️ 1.01x slower	12.5±0.31ms	12.6±0.33ms
`crcVeryStale/snapshotLatest`	✅ 1.00x	19.0±0.49ms	19.0±0.48ms
`partitioned/readMetadataLatestPredicate/serial`	☑️ 1.03x slower	61.8±2.27ms	63.5±3.81ms
`v1Checkpoint/readMetadataLatest/serial`	✅ 1.02x faster	13.0±0.18ms	12.8±0.13ms
`v1Checkpoint/snapshotLatest`	✅ 1.01x faster	918.3±24.59µs	909.1±26.41µs
`v2Checkpoint/readMetadataLatest/serial`	✅ 1.01x faster	14.9±0.15ms	14.7±0.26ms
`v2Checkpoint/snapshotLatest`	✅ 1.00x	882.0±23.10µs	881.8±25.37µs
`wideSchemaJsonStats/readMetadataLatestPredicate/serial`	✅ 1.00x	77.4±2.31ms	77.2±1.64ms
`wideSchemaJsonStats/snapshotLatest`	🚧 1.04x slower	2.4±0.05ms	2.5±0.06ms
`wideSchemaStructStats/readMetadataLatestPredicate/serial`	☑️ 1.01x slower	34.7±0.94ms	34.9±0.87ms
`wideSchemaStructStats/snapshotLatest`	✅ 1.00x	2.3±0.04ms	2.3±0.06ms

_{Legend: 🚀 ≥1.15x faster · ✅ faster or unchanged · ☑️ ≤1.03x slower · 🚧 1.03x-1.15x slower · ❌ ≥1.15x slower}
_{Commit: 2e1694f · Trigger: auto-push · Tags: base · Updated: 2026-06-22 23:54 PDT}

… mapping The 3-way rewrite of try_assign_field_column_mapping flows straight through apply_schema_operations, so the same preserve/fill/assign matrix applies to ALTER as to CREATE. These tests pin that contract end-to-end through the alter -> snapshot read roundtrip: - both id + physicalName preserved when fully supplied (in name and id modes) - id allocated when only physicalName is supplied - physicalName filled when only id is supplied - supplied id less than the existing maxColumnId is accepted as long as it doesn't collide. This matches Spark and diverges from the Java Kernel proposal in delta-io/delta#4520. - supplied id that collides with an existing field is rejected Refs delta-io#2377.

…ta-io#2501) ## Stacked PR Use this [link](https://github.com/delta-io/delta-kernel-rs/pull/2501/files) to review incremental changes. - [stack/preserve-cm-feat](delta-io#2500) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2500/files)] [MERGED] - [**stack/preserve-cm-create-tests**](delta-io#2501) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2501/files)] (this PR) - [stack/preserve-cm-alter-tests](delta-io#2502) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2502/files)] --------- ## What changes are proposed in this pull request? Integration tests pinning column-mapping (CM) metadata preservation end-to-end through the public create-table -> snapshot-reload path. The underlying preserve/fill/reject logic is unit-tested in `table_features::column_mapping` and `schema` (`make_physical`); these tests cover the public wiring and on-disk persistence: - complete CM metadata (id + physicalName) preserved verbatim - physicalName filled with col-<uuid> when only id is supplied - id allocated when only physicalName is supplied - sparse preserved ids (1, 5, 100) seed new allocations above the max (101+) - CM metadata preserved under nested struct fields Also loosens `assert_column_mapping_config` to compare maxColumnId against the schema's actual largest mapping id rather than the total field count. The old invariant only held when the kernel assigned every id itself; preserving connector-supplied ids breaks it. Refs delta-io#2377. ## How was this change tested? New integration tests in kernel/tests/integration/create_table/column_mapping.rs, run with `cargo test -p delta_kernel --all-features --test integration`. fmt, clippy, and the full suite pass. Co-authored-by: Sanuj Basu <sanujbasu@users.noreply.github.com>

github-actions Bot assigned sanujbasu Apr 30, 2026

This was referenced Apr 30, 2026

test: CREATE TABLE tests to validate column mapping preservation #2501

Merged

feat: preserve pre-existing column mapping metadata on writes #2500

Merged

sanujbasu changed the title ~~Add ALTER TABLE ADD COLUMN integration tests for preserve/fill column mapping~~ test: add ALTER TABLE ADD COLUMN integration tests to validate the preservation of existing column mapping Apr 30, 2026

sanujbasu force-pushed the stack/preserve-cm-alter-tests branch 2 times, most recently from 7ef58b4 to d5064b7 Compare May 1, 2026 08:54

sanujbasu force-pushed the stack/preserve-cm-alter-tests branch 2 times, most recently from e1920fa to 905d9d2 Compare May 12, 2026 10:19

sanujbasu requested review from DrakeLin and dengsh12 May 12, 2026 21:07

sanujbasu force-pushed the stack/preserve-cm-alter-tests branch from 905d9d2 to e45b286 Compare May 14, 2026 01:14

sanujbasu marked this pull request as ready for review May 14, 2026 01:15

sanujbasu added the Area: Create table label May 14, 2026

sanujbasu force-pushed the stack/preserve-cm-alter-tests branch from e45b286 to 6d4aa0e Compare June 19, 2026 01:19

sanujbasu changed the title ~~test: add ALTER TABLE ADD COLUMN integration tests to validate the preservation of existing column mapping~~ test: add ALTER TABLE tests validating existing column mapping annotations Jun 22, 2026

sanujbasu force-pushed the stack/preserve-cm-alter-tests branch from 6d4aa0e to 4f2974c Compare June 22, 2026 23:02

sanujbasu force-pushed the stack/preserve-cm-alter-tests branch from 4f2974c to 2e1694f Compare June 23, 2026 06:45

sanujbasu added the Area: Alter table label Jun 23, 2026

DrakeLin approved these changes Jun 23, 2026

View reviewed changes

dengsh12 approved these changes Jun 23, 2026

View reviewed changes

sanujbasu added this pull request to the merge queue Jun 23, 2026

Merged via the queue into delta-io:main with commit 5d99205 Jun 23, 2026
30 of 32 checks passed

sanujbasu deleted the stack/preserve-cm-alter-tests branch June 23, 2026 18:32

Navigation Menu

Uh oh!

test: add ALTER TABLE tests validating existing column mapping annotations - #2502

test: add ALTER TABLE tests validating existing column mapping annotations#2502
sanujbasu merged 1 commit into
delta-io:mainfrom
sanujbasu:stack/preserve-cm-alter-tests

sanujbasu commented Apr 30, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Apr 30, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 19, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

sanujbasu commented Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes are proposed in this pull request?

How was this change tested?

Uh oh!

codecov Bot commented Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions Bot commented Jun 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmark results: ✅ Pass

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sanujbasu commented Apr 30, 2026 •

edited

Loading

codecov Bot commented Apr 30, 2026 •

edited

Loading

github-actions Bot commented Jun 19, 2026 •

edited

Loading