spock_apply_heap_insert() remoteslot uses generic TTSOpsVirtual, breaks INSERT->UPDATE conflict resolution on index-organized AMs (OrioleDB)

[mrayva]

Summary

spock_apply_heap_insert() creates its remoteslot with a generic TTSOpsVirtual slot. That's fine for a plain insert, but when the incoming INSERT conflicts with an existing row (found=true), Spock converts it into an UPDATE and applies it via ExecSimpleRelationUpdate() using that same remoteslot — which then hits the same AM-native-slot requirement that spock_apply_heap_update() already handles correctly, but spock_apply_heap_insert() does not.

Root cause

For table AMs with no physical tuple identifier of their own — index-organized AMs such as OrioleDB, which report ROW_REF_ROWID from table_get_row_ref_type() — ExecUpdateIndexTuples() (reached from ExecSimpleRelationUpdate()) needs to retrieve the row's AM-native identifier via slot_getsysattr(RowIdAttributeNumber) on the slot being updated. A TTSOpsVirtual slot can never satisfy that. table_tuple_update() has no return-value mechanism (unlike table_tuple_insert(), which can hand back a substituted slot) for an AM to fix this up after the fact, so the slot must already be AM-native going in.

spock_apply_heap_update() (src/spock_apply_heap.c) already does this correctly:

remoteslot = table_slot_create(rel->rel, &estate->es_tupleTable);

But spock_apply_heap_insert() still does:

remoteslot = ExecInitExtraTupleSlot(estate,
                                    RelationGetDescr(rel->rel),
                                    &TTSOpsVirtual);

and reuses this same remoteslot in its found=true branch, which ends up in the identical ExecSimpleRelationUpdate() call as the plain-UPDATE path.

Symptom

On an INSERT/INSERT conflict (both nodes insert the same primary key concurrently) against an OrioleDB table, the conflict-resolution logic itself decides correctly (e.g. apply=1 resolution=ApplyRemote), but applying that decision then fails — observed as a crash inside the apply worker with a stray-byte/protocol-desync-looking error (unknown action of type <garbage>, see the related issue on the exception-replay path), and on retry the transaction is discarded rather than applied, leaving the two nodes permanently diverged for that row. A plain (ERROR: cannot retrieve a system column in this context / assert, depending on build) is also directly reachable through the same slot mismatch.

Reproduction

Two-node Spock cluster with an OrioleDB table on both sides, bidirectional subscriptions. Insert the same primary key concurrently from both sides (disable both subscriptions, insert different rows with the same PK on each node, then re-enable both subscriptions so the conflict is detected on apply):

-- both subscriptions disabled
-- node1
INSERT INTO oriole_items (id, name) VALUES (401, 'from-node1');
-- node2
INSERT INTO oriole_items (id, name) VALUES (401, 'from-node2');
-- re-enable both subscriptions

Suggested fix

remoteslot = table_slot_create(rel->rel, &estate->es_tupleTable);

in place of the ExecInitExtraTupleSlot(..., &TTSOpsVirtual) call in spock_apply_heap_insert(), matching the pattern already used in spock_apply_heap_update(). slot_store_data() (used to populate this slot for the plain-insert path) calls ExecStoreVirtualTuple(), which works generically regardless of the slot's underlying AM type, so this is safe for the plain-insert case too. Verified fix converges correctly across 10+ rounds of concurrent INSERT/INSERT conflicts against OrioleDB; patch available on request.

Environment

• Spock built from main (reports as version 6.0.0)
• PostgreSQL 18.6, OrioleDB built against the same base
• Two-node bidirectional cluster

Related

• ERROR: virtual tuple table slot does not have system attributes orioledb/orioledb#64 — same underlying symptom class ("slot lacks system attributes/identity OrioleDB needs").
• Assert comparing length of variable-length primary key when applying on logical subscriber with non-default collation orioledb/orioledb#662 — a different but adjacent crash in the native (non-Spock) subscriber apply path, also reached via table_tuple_update() on an OrioleDB table.
• Bi-directional logical replication. Loop ERROR: Key already exists in apply worker orioledb/orioledb#672, #665 — native (non-Spock) bidirectional logical replication apply-worker crash loops on OrioleDB tables; same general fragility area, different specific mechanism.