Lazy conjugation at evaluation: retrieval-time CanonTransform, Re/Im eval nodes, Adjoint IR retired - #603
Draft
kshitij-05 wants to merge 24 commits into
Draft
Conversation
…oint IR retired from binarize; product-level conj hoisting Plan tasks 4+6+7(product half) as one green unit (see the deviations section appended to the plan doc): the conjugation marker composes as a PURE conj bit (slot deltas belong to the canonicalizer fold alone), collect_tensor_factors re-materializes denoted spellings for TN identity and slot counting, child hashes carry a conj/swap structural salt, and uniformly conjugated products hoist the conjugation onto the node transform (the Kramers-tracing cache-reuse mechanism). binarize(Tensor) is a plain leaf; the PR-1 refusal gates and the make_adjoint_node channels are gone; IR-shape and boundary tests reblessed to the transform model.
…lling normalization
The ToT branch consumes the canonicalize_slots conjugated_tensors report
(PR-1's field gains its consumer): fold ON, the canonical labeling's
swapped-orientation report composes {conj, braket_swap} and the stored
spelling is respelled canonical, so flat and ToT leaves share one
contract. '⁺'-label and marker channels factored into
normalize_leaf_spelling, used by both branches.
…-scalar wraps
The hoist decision is taken over the factor/summand node transforms and
strips the conj salt from the child-hash stream, so a uniformly
conjugated product, sum, or real-scalar-wrapped term hashes onto its
unconjugated counterpart's slot with {conj} on the node -- pairwise
intermediates included (every prefix of a uniform factor list is
uniform). Mixed marks keep their structural salts. Pinned by
conj_hoisting_structural_identity.
CanonTransform moves to its own header (canon_transform.hpp) so the result layer sees it without depending on eval_expr. Base virtual with throwing default; ResultScalar conjugates non-arithmetic values; the TA backend (flat + ToT) fuses phase * conj * relabel in one expression, conj elided for real numeric_type exactly as adjoint(). The trivial transform is the caller's short-circuit, mirroring apply_phase. BTAS/tapp implementations land with the evaluate switch (T9).
The stack machine's apply_phase generalizes to the full transform: a leaf's result is served as its DENOTED value (the leaf evaluator serves the canonical spelling; the hand-up applies the transform, and finish_phase_b's involution store keeps the cache canonical), the NeedLeftAdj stage, EvalOp::Adjoint, is_adjoint, the export Adjoint case and Result::adjoint (all four backends) are gone, and every aliasing check keys on canon_transform().trivial(). BTAS/tapp gain fused apply_transform implementations; symmetrize/antisymmetrize wrappers use the DENOTED bra rank. ToT leaves carry array-faithful Nested (outer;inner) canon_indices via tot_indices. The TA test fixture adopts the canonical-serving contract (keys and stored arrays are canonical- spelling shaped; literal spellings are served through the leaf's transform); IR-shape and end-to-end expectations reblessed to the transform model. Both suites green (debug 6558; TA-enabled 159612 assertions).
x and x^* (and b^n vs conj(b^n)) share one cache slot: the unmarked spelling is stored and hashed, the conj bit rides the CanonTransform, applied on retrieval by ResultScalar::apply_transform and re-materialized as a marker by the export layer's denoted_expr.
The left-fold's factor prefixes hoist independently: a uniformly
conjugated prefix strips its factors' conj salts (per-PREFIX, keeping
C.C^* vs C^*.C order-insensitive) and records {conj} on its node, so
the (A^*.B^*) intermediate of A^*.B^*.C is a cache hit on the A.B slot
while mixed roots stay identity-distinct. conj_eval_cache_reuse pins
the whole contract with a counting yielder on random complex data:
whole-term partner, \mathcal{T}-shaped sum of products, and the
buried-intermediate CSE -- zero new leaf yields plus numeric equality
with the conjugated reference in every shape. Also purges Theta in
favor of \mathcal{T} notation throughout.
Owner decision: fold_conjugate_pairs_of_real_sum is fragile (reality assertion, unfolded difference pairs) and retires in favor of the exact fold_conjugate_pairs, which needs eval-side Re/Im nodes -- so those move from Plan B into this PR, with the MPQC call-site switch and smoke re-certification.
The conjugate-pair fold emits Constant(2) * RealPart(s); Re/Im are
projections (not involutions), so they cannot ride in CanonTransform and
become unary IR nodes over the shared inner subtree (Constant{1} sentinel
right child). Node hash = inner's salted hash + op, so Re(s), Im(s) and
bare s occupy distinct slots while the inner stays shared.
Result::real_part()/imag_part() virtuals (default-throwing; ResultScalar
serves both backends' scalars); linearize_eval_node reconstructs the
wrappers; export throws loudly pending its Plan-B support.
The parked default flips now that eval ingests Re/Im nodes. Hardening: simplify() folds only fully c-number expressions (operator-carrying intermediates head back into Wick, which does not ingest the wrappers); the fold merges Re/Im-wrapped summands at entry and exit via Re(x*) = Re(x) / Im(x*) = -Im(x) over canonical representatives, so wrappers from different passes combine and cancel exactly; Re/Im canonicalize their inner in place (+ real-scalar hoist via the byproduct contract) and Product invalidates its memoized hash when a subfactor mutates; has_tensor sees through the wrappers. Tests: fold-shape assertions compare canonically, UCC energy term counts updated for the folded pairs (46->23, 20->14, 74->41), test_mbpt_cc's shadowing has_tensor lambda delegates to the core one, re_im_evaluation pins its symbolic environment.
Fragile back-compat variant: it asserts (unverifiably) that the sum's
value is real and silently leaves {s, -s*} difference pairs unfolded.
Use fold_conjugate_pairs. Removal deferred until MPQC's CC energy path
switches over.
get_unique_indices: a scalar-valued Re/Im wrapper has no external indices (the CSV/PNS pipeline feeds folded energy expressions through it). binarize_re_im: the wrapper node carries a Variable placeholder like every other non-leaf scalar node -- binarize(ResultExpr)'s scalar path set_labels the root's Variable, and the previous RealPart-typed expr made that a wrong-type downcast (UB under an IGNORE-assert build).
An opaque wrapper came back untouched from optimize_impl, leaving its inner product to evaluate in naive left-to-right order (measured 14.4 GB vs 1.7 GB peak RSS on a Kramers-CSV MP2 energy with three folded terms). Optimize the wrapped expression and re-wrap; regression-tested for Re, Im, and a wrapped summand inside a Sum.
Adjoint nodes are gone (retrieval-time CanonTransform); the report's full_expr now renders the RealPart/ImagPart unary wrappers instead.
…ia lazy TA conj, T20) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AkZPie5Up85F78r94P4jQs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Conjugation PR 2 of the series; stacked on #602 (
kshitij/feature/conjugation-symbolic), to be retargeted tomasterwhen #602 merges.What
Serve conjugation at evaluation time through the retrieval transform instead of an
AdjointIR node.EvalExpr::canon_phase_generalizes toCanonTransform {phase, conj, braket_swap}: the map from the cached canonical value to the value this node denotes. Excluded from the node's slot hash, as the phase already was.Result::apply_transform(transform, annot)(one backend virtual; the TA implementation fuses relabel,.conj()and scale) replacesapply_phaseon retrieval.EvalOp::Adjoint, theNeedLeftAdjstage, theConstant{1}sentinel andResult::adjointare retired.fold_conjugate_brakethas one setting everywhere), starred Nonsymm leaves and⁺-labeled adjoints are served via the transform instead of throwing, ToT leaves fold viaconjugated_tensors.conj(A·B)andA·Bshare intermediates.EvalOp::RealPart/EvalOp::ImagPartnodes;simplifyfoldsA + A*into2 Re Aby default (FoldConjugatePairs::Yes, hardened);fold_conjugate_pairs_of_real_sumdeprecated. The optimizer sees through the Re/Im wrappers.Verification
[conjugation],[eval_expr], uniform-conjugate reuse tests) plus the existing suite; run withSEQUANT_ASSERT_BEHAVIOR=THROW.Known limitation
A Re-wrapped ToT (CSV) energy summand evaluates through a materializing nested einsum instead of the trace-mode scalar reduction (14 GB on dch). MPQC therefore ships the CSV energy fold opt-in; the fix (route the wrapper's inner root through the ResultExpr scalar-head treatment) is the recorded follow-up T20 in
doc/dev/plans/2026-09-01-lazy-conj-eval-plan.md.Design:
doc/dev/specs/2026-09-01-lazy-conj-eval-design.md.🤖 Generated with Claude Code
https://claude.ai/code/session_01AkZPie5Up85F78r94P4jQs