Replace polymarket positions input with ERC1155 reconstruction

[0xRobin]

Replace polymarket_polygon.positions_raw's upstream dependency on tokens_polygon.balances_daily_agg_base (which is built from a snapshot source that carries stale balances when burn events are missed) with a new model, polymarket_polygon.positions_balances_repro, that reconstructs daily Polymarket CTF balances directly from on-chain erc1155_polygon.evt_TransferSingle and evt_TransferBatch events.

Linear: CUR2-2534

Changes

• New incremental model polymarket_polygon.positions_balances_repro:
  • Computes per-day net delta per (address, token_id) from Polymarket CTF transfer events.
  • Filters days where every transfer has value = 0 (no-op transfers in batches), but keeps intraday round-trips with non-zero amounts so closure markers are emitted.
  • Cumulative int256 running sum, clamped to [0, uint256_max], cast to uint256. No double-precision intermediate.
  • Forward-fills each balance row to the day before the next change, anchoring at the closure day for balance = 0.
  • Incremental seeds from {{ this }} over the prior window, aggregated by last_updated (not day) so last_updated is preserved across forward-filled rows.
  • In incremental runs the forward-fill fill_start is clipped to the window start so cold pairs don't expand their full history before the predicate filters it.
  • For closed positions (balance_raw = 0) the fill_end is capped at the closure day so we don't expand [closure_day, current_date - 1] only to be dropped by the final WHERE.
• polymarket_polygon.positions_raw now reads from the new model instead of balances_incremental_subset_daily(...). Output schema unchanged.
• _schema.yml: adds the new model with dbt_utils.unique_combination_of_columns (scoped to last 14 days for CI runtime) and not_null tests on key columns.

CI vs prod comparison (last day, run 26572354448)

	rows
balance_matching (same key, same balance_raw)	69,369,909 (99.93 %)
balance_different (same key, different balance_raw)	10,050
ci_only (in CI, missing in prod)	18,160
prod_only (in prod, missing in CI)	40,605

Over the last 7 days the aggregate balance sums agree to within 0.001 % (16.45607777e18 vs 16.45625946e18 raw).

last_updated parity on the same 68.2 M common rows: 99.993 % identical, 4,779 differ. All observed differences are negative drift (CI's last_updated is later than prod's), consistent with prod's snapshot source missing intermediate change events.

Why the residual diffs are expected and acceptable

I sampled rows from each diff bucket and traced the underlying ERC1155 events:

• balance_different: legacy balances_polygon_0002 snapshots miss some burn events, leaving stale positive balances that diverge from on-chain reality.
  • Example: 0xc4c41cd12f991e76ceeff92b531cd74c02497bc2 / token 1119...477. Prod = 10_527_110_704. Sum of all on-chain deltas for this pair = 1_316_636_682_369 (last event 2022-03-26). The new repro matches the on-chain sum exactly. Prod has been wrong for ~4 years.
• prod_only: 100 % have balance_raw > 0, all are forward-fills (last_updated < day), 39,829 of 40,605 have last_updated older than 90 days. These are the same class of stuck legacy snapshot rows. The repro correctly omits them because the on-chain net delta is zero.
• ci_only: 100 % have balance_raw > 0, all are forward-fills of real on-chain positions that the legacy snapshot source never indexed. The repro correctly includes them.

Net effect: the repro fixes ~69k stale/missing rows per day relative to the legacy table while matching the remaining 99.9 % exactly, and additionally restores accurate last_updated values for pairs whose intermediate changes were missed by the snapshot source.

Performance

CI initial build: ~20 min, ~9.4 B rows materialized; incremental merge: ~3 min, ~200 M rows. Same order as the legacy model.

Validation

• CI green, including the in-CI dbt_utils.unique_combination_of_columns and not_null schema tests on the new model.
• Manual row-level diff against prod for the last day (above) and aggregate diff for the last 7 days.
• last_updated parity check (above) confirms the seed-by-last_updated fix prevents drift on inactive pairs (~99 % of pre-window pairs are forward-fills with drift up to 5.3 years under the naive seed-by-day approach).
• Traced sample rows from each diff bucket back to the underlying ERC1155 transfer events to confirm the repro matches on-chain truth.

What I removed from the original Cursor-agent draft

• The singular equivalence test (tests/polymarket_polygon_positions_balances_repro_equivalence.sql). It compared the repro against balances_incremental_subset_daily(...) — which is the legacy table we are intentionally moving off and is itself the source of the divergences above. Any threshold tight enough to be useful would have made CI red on legitimate fixes; loose enough not to was useless. The schema-level uniqueness / not-null tests stay as the in-CI guardrails.
• The double precision cast in the running balance computation. sum(int256) OVER (...) works natively and avoids the rounding risk for large positions.
• The two-stage daily_flows → daily_deltas aggregation and the max(block_number) / max(block_time) columns that were computed but never emitted.

---

quick links for more information:

• README.md
• spellbook docs
• CONTRIBUTING.md

[0xRobin]

@BugBot review

[cursor]

PR Summary

Medium Risk
Changes the upstream source for user position balances (large incremental table, ~99.9% row match to prod with intentional fixes for stale/missing legacy rows); downstream positions semantics depend on this pipeline but schema is preserved.

Overview
Polymarket positions now derive daily CTF balances from on-chain ERC1155 TransferSingle / TransferBatch events instead of tokens_polygon.balances_daily_agg_base (snapshot-based balances that could miss burns and leave stale rows).

A new incremental model polymarket_polygon_positions_balances_repro builds per-day signed deltas, cumulative int256 balances (clamped, no double intermediates), forward-fills through the day before the next change, and filters zero-value no-op transfer days. polymarket_polygon_positions_raw reads from that model; its output shape (balance, last_updated, etc.) is unchanged.

Tests: _schema.yml documents the repro model and scopes unique_combination_of_columns on both raw and repro to the last 14 days for CI; repro keys get not_null checks.

^{Reviewed by Cursor Bugbot for commit 3d02747. Configure here.}

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 3d02747. Configure here.}