Consensus elongation - #308
Draft
arnabmutsuddy wants to merge 58 commits into
Draft
Conversation
…idation tree
Phase A.1+A.2 of the trna_charging_final port from CovertLab/vEcoli@trna_charging_final
(HEAD 330ee3f4). Lays the groundwork required before KineticTrnaChargingModel
(polypeptide_elongation.py:2198 upstream) and the Cython kernel _trna_charging.pyx
can be brought into v2ecoli.
* relation.py: extend the 145-line stub to 1421 lines from upstream, adding
_build_codon_sequences, _build_codon_based_translation,
_build_codon_dependent_trna_charging, _build_trna_charging_kinetics
(plus optimize_trna_charging_kinetics, get_constants, assign_K_T helpers).
Imports rewritten from upstream-style (reconstruction.*, wholecell.*) to
v2ecoli's nested-ParCa absolute style.
* knowledge_base_raw.py: register 6 new flat-file loader entries
(trnas, trna_charging_kinetics, trna_charging_kinetics_curated,
optimization/trna_charging_kinetics_constants,
optimization/trna_charging_kinetics_solutions,
optimization/trna_synthetase_dynamic_range). Legacy trna_data/* paths kept
for now; Phase D will remove them once all downstream references are migrated.
* Flat data: 6 new ParCa TSVs (incl. 5.1 MB trna_charging_kinetics_solutions.tsv
with 5680 fitted parameter rows) and the full upstream validation/ecoli/flat/
tree (24 files) — needed because ValidationDataRawEcoli enumerates them in
LIST_OF_DICT_FILENAMES.
* validation/ecoli/{validation_data,validation_data_raw}.py: 613 lines total,
ported with import path rewrites. Provides ValidationDataEcoli with the new
trna_synthetase_kinetics, jakubowski1984, and dong1996 datasets used by
KineticTrnaChargingModel and its analyses.
* workspace/investigations/trna-charging-final/audit.md: file-by-file audit of
the trna_charging_final diff vs the existing v2ecoli port (status legend,
line counts, destination decisions, sizing) plus Phase A progress log.
Runtime-verified: ValidationDataRawEcoli loads 14 TSVs cleanly;
KnowledgeBaseEcoli loads in 1.6s with 44 trnas, 20 charging-kinetics rows,
5680 optimization solutions, 40 dynamic-range rows accessible via the new
attribute paths.
Not yet landed (Phases B-E): _trna_charging.pyx Cython kernel → NumPy+numba,
KineticTrnaChargingModel class, kinetic_charging_baseline composite arch,
library/sim_data and process deltas, full ParCa rerun, behavior/parity gates,
multigeneration reports.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Pairs with workspace/investigations/trna-charging-final/audit.md. Captures branch state (last commit 885df04), locked architectural decisions, venv setup workaround for vivarium-dashboard, the ordered remaining-task table (#2-#13) with sizing, suggested session boundaries, and a prompt template the user can paste into a new session to resume. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
#6) Brings 5 ParCa dataclasses up to trna_charging_final's required state so the Relation tRNA methods (committed in 885df04) and the forthcoming KineticTrnaChargingModel can find the data they expect. * translation.py: cleavage_of_initial_methionine bool column on monomer_data (used to know which proteins start at M+1 after fMet cleavage). * molecule_groups.py: new codons (4^3 minus UAA/UAG stops), initiator_trnas (4 fMet-tRNAs), elongator_trnas (2 elongator Met-tRNAs) groups consumed by Relation._build_codon_dependent_trna_charging and Mass._build_trna_data. * simulation_data.py: codon_read_rate {} init paralleling translation_supply_rate; populated by ParCa, read by Relation.optimize_trna_charging_kinetics. * transcription.py: anticodon (U3) field on cistron_data populated from raw_data.rnas[*]["anticodon"]; aa_from_trna dtype float->int matching the upstream "Integer mapping matrix for tRNAs to amino acids" change. * growth_rate_dependent_parameters.py: rewrite Mass._build_trna_data to read from the new single trnas.tsv (anticodon-based Kurland-WCM mapping with optional sequence-disambiguation via Bio.Seq.reverse_complement_rna), replacing the legacy multi-file trna_data/trna_ratio_to_16SrRNA_* loader. Output matrix transposes to (n_trnas, n_growth_rates); get_trna_distribution adjusted accordingly. Two upstream files were intentionally **skipped** (audit doc explains): two_component_system.py (reverses upstream-master compartment-tag validation that postdates trna_charging_final) and scripts/nca/run_all.py (drops raw- string regex prefixes, re-introducing Python-3.12 DeprecationWarnings). Neither is tRNA-charging logic. Runtime-verified: KnowledgeBaseEcoli loads in 1.5s with anticodon field populated on every rnas row; new trna_data attributes accessible. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…tion The kernel is 11 mostly-independent functions, not one monolith. Splitting along function boundaries lets each session land as one logical commit with its own parity gate, and isolates the three hard kernels (reconcile_via_ribosome_positions, reconcile_via_trna_pools, get_elongation_rate) from the 8 easy ones. 2a is the gatekeeper: building the upstream Cython once and dumping golden inputs/outputs is what makes 2b-2e mechanically verifiable instead of "looks right by inspection." Sizing: 5 sessions of 2-4 hr each, replacing the original "1-2 day" single-developer estimate. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…k 2a)
Sets up everything 2b-2e need to verify the NumPy+numba port matches the
upstream Cython _trna_charging.pyx behavior.
New files
* workspace/investigations/trna-charging-final/capture_kernel_golden.py:
one-shot script that imports the built upstream Cython kernel
(compiled in vEcoli_trna/.venv via `setup.py build_ext --inplace`) and
dumps 25 input/output cases across 9 functions to a gz-compressed JSON.
Test-case inputs lifted from upstream wholecell/tests/utils/
test_trna_charging.py plus larger seeded inputs for reconcile_via_*.
* tests/fixtures/trna_charging_kernel_golden.json.gz (1.7 KB gz / 12 KB raw):
the captured golden. Includes provenance metadata (upstream SHA
330ee3f4, capture time, platform, RNG identifier) and a libc-rand note
flagging the platform-dependency the port has to work around.
* v2ecoli/processes/polypeptide/kinetic_charging_kernel.py: module
skeleton with the seed/randint_below RNG wrapper (numpy.random.RandomState
backed, independent of the global numpy.random state) and 10
NotImplementedError stubs ready for 2b-2e. Module docstring documents
the per-RNG parity policy.
* tests/test_kinetic_charging_kernel_scaffold.py: 10-test scaffold gate.
Verifies the golden round-trips with all 9 functions covered, the RNG
is deterministic / divergent / global-state-independent / raises when
unseeded, and every stub has the documented signature.
Verified
pytest tests/test_kinetic_charging_kernel_scaffold.py -> 10 passed in 5.0s
pytest wholecell/tests/utils/test_trna_charging.py -> 15 passed in 0.44s
(upstream Cython tests, run in /Users/arnabmutsuddy/projects/vEcoli_trna/vEcoli)
RNG policy
The kernel uses libc rand(). macOS vs glibc rand() differ; our numpy
RandomState is yet a third RNG. So byte-for-byte parity vs the upstream
Cython is impossible by design. The scaffold's policy: deterministic
functions (8 of 11) assert exact-equality vs the captured libc-rand
golden because they don't touch the RNG; stochastic functions (3) will
get a sibling numpy-randomstate golden captured at port time in 2c/2d
and assert byte-identity against THAT, with the libc-rand golden kept
only as a sanity-shape check. Documented in detail in audit.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replaces the NotImplementedError stubs in v2ecoli/processes/polypeptide/kinetic_charging_kernel.py for the 7 functions that don't touch the RNG. All ported as @njit(error_model="numpy") matching the precedent set by polypeptide/kinetics.py. Ported * get_initiations: count ribosomes with elongations>0 and lengths==0. * get_codon_at: bounded sequence-table lookup, returns -1 outside range. * get_candidates_to_C / get_candidates_to_N: directional scans for codon matches across ribosomes; cleaner signature than upstream (dropped 4 scratch parameters Cython used to recycle locals). * select_candidate: index of the r-th match. Discovered during port that this function is purely deterministic — upstream's `r = rand() % candidates` happens at the caller, not inside select_candidate. So the RNG seam is first exercised in 2c, not here. Documented in the docstring. * is_initial_state: int32 elementwise equality predicate. Upstream Cython flags it as unused at build time; ported anyway for API symmetry. * get_codons_read: codon-usage histogram across ribosome extents. Parity gate * tests/test_kinetic_charging_kernel.py — 7 parity tests, one per function, iterating every relevant case from tests/fixtures/ trna_charging_kernel_golden.json.gz and asserting bit-identical output vs the upstream Cython captures. Plus test_is_initial_state_local_cases (no golden coverage since upstream doesn't call it) and a test_2b_covers_every_relevant_golden_case belt-and-suspenders gate. * tests/test_kinetic_charging_kernel_scaffold.py: updated the test_stubs_raise_not_implemented_until_filled_in test to point at the still-stubbed 2c/2d/2e functions; it now also asserts the per-task marker strings in the NotImplementedError messages. Results pytest tests/test_kinetic_charging_kernel{,_scaffold}.py -> 18 passed, 3 skipped, 3.93s including numba JIT warm-up. Notes for 2c/2d Numba can nest @njit calls, confirmed in this commit: get_candidates_to_C/N and select_candidate all call our jitted get_codon_at directly. 2c can call the 2b-ported functions the same way without going through Python. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Plain-Python orchestration calling the 2b @njit helpers (get_codon_at, get_candidates_to_C/N, select_candidate). RNG draws route through kernel.randint_below. Two non-obvious upstream behaviors preserved (documented in the function docstring + audit.md): 1. disagreements_remaining state leaks across attempts. Initialized once at function entry, set False by phase 1 exit and never reset, explicitly reset to True before phase 2's while. So on attempt 2+, phase 1 is *skipped*. test_reconcile_attempts_threshold depends on this for its [10, 15] expected output -- without the leak, the algorithm would oscillate forward/backward on non-convergent inputs. 2. Phase 2 has no exhausted array. Surplus codon implies the codon exists somewhere in the consumed-so-far range, so get_candidates_to_N is guaranteed to return candidates >= 1. Parity gates (two complementary): * Byte-identity vs a committed numpy-RandomState golden (tests/fixtures/trna_charging_kernel_numpy_randomstate_golden.json.gz, 1.1 KB gz / 5.9 KB raw). Regenerated by workspace/investigations/trna-charging-final/capture_numpy_randomstate_golden.py -- which is also wired for 2d/2e to extend later. Detects regressions in the port at the byte level for a fixed RNG. * Algorithmic-invariant checks against the libc-rand golden. Bytes differ (different RNGs pick different ribosomes) but these are RNG-independent: - kinetics_codons never mutated - sequence_codons, elongations non-negative - conservation: delta(elongations.sum()) == delta(sequence_codons.sum()) - convergence parity: when upstream reached compromise=0, v2ecoli does too Sanity: reconcile_via_ribosome_positions/equal is byte-identical libc <-> numpy (no RNG calls); attempts_threshold reaches same final sequence_codons=[10,15] and same elongations.sum()=25 despite different per-ribosome trajectories. Confirms the algorithm's convergence is RNG-invariant. Results: pytest tests/test_kinetic_charging_kernel{,_scaffold}.py -> 20 passed, 2 skipped (for 2d/2e), 3.78s Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Same shape as 2c: ~95 LOC pure-Python orchestration calling randint_below
for RNG draws. Two-branch structure picks how to "pull back" the Kinetic
Model by undoing one codon read at a time:
* Free-tRNA branch: a free tRNA reading this codon returns to charged form.
Net: free_trnas[i] -= 1, charged_trnas[i] += 1,
codons_to_trnas_counter[i, codon] -= 1, kinetics_codons[codon] -= 1.
* Charged-tRNA branch (no free): undo the most recent charging *and* the
codon read. Net: chargings[i] -= 1, amino_acids_used[aa] -= 1,
codons_to_trnas_counter[i, codon] -= 1, kinetics_codons[codon] -= 1.
Free/charged abundances unchanged because the tRNA goes free -> charged ->
free, ending where it started.
Key difference from 2c: this function MUTATES kinetics_codons (decrementing
one entry per pick). sequence_codons is read-only.
Parity (same two-gate pattern as 2c):
* Byte-identity vs the numpy-RandomState golden -- refreshed to 10 cases
(was 8, the 2 trna_pools placeholders are now filled).
* Algorithmic invariants vs libc-rand golden:
- sequence_codons never mutated
- per-tRNA conservation: free_trnas[i] + charged_trnas[i] unchanged
- post-loop: kinetics_codons[c] <= sequence_codons[c] for all c
- chargings, amino_acids_used, codons_to_trnas_counter >= 0
- kinetics_codons FINAL state is RNG-invariant -- the loop runs until
disagreements=0 with one decrement per iteration, so the total
decrement is fully determined by initial disagreements. Asserted as
byte-identity vs upstream's kinetics_codons_out.
Sanity: both libc-rand test cases (use_free_trna, forward_undo_charging)
yield byte-identical libc <-> numpy outputs. Reason: the test inputs only
allow one viable tRNA pick per iteration, so the RNG sequence has nothing
to distinguish between.
Results: pytest tests/test_kinetic_charging_kernel{,_scaffold}.py
-> 22 passed, 1 skipped (only 2e left), 3.83s
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…l port The 11th and last upstream Cython kernel function lands as a pure-deterministic @njit binary search in v2ecoli/processes/polypeptide/kinetic_charging_kernel.py (~110 LOC). No RNG, no mutating writes to inputs. Algorithm: binary-search over column count to find the number of timesteps whose measured elongation rate is closest to `target`. Each step counts non-(-1) entries in sequences[:, :col], compares to target, and bisects [lower, upper]. Termination: the search proposes a col already visited. The post-search "snap to time-step boundary" logic returns col // time_int (when best_col is even) or the floor/ceil rounded step whose rate is closer to target. Two upstream quirks preserved verbatim for byte-identity: * The `best_col % 2 == 0` heuristic is only strictly correct for time_int<=2. For higher integer time values it can return the wrong rounded result. We match upstream; documented in the function docstring. * When the binary search hits rate==target and breaks without incrementing index, the post-loop "find best" sweep `for i in range(index)` skips the just-written entry. Upstream's Cython relies on `cdef int i = 0` initialization to make this case work via the degenerate range(0); we initialize best_i=0 to match. Parity gate * test_get_elongation_rate_parity: bit-identical vs upstream Cython for the captured case. Passes. * test_reconcile_seed_propagates_to_kernel_output replaces upstream's test_reconcile_different_seeds_different_results. Surprising finding while writing this test: with our numpy.random.RandomState, the upstream big_seed_12345 and big_seed_54321 inputs converge to *identical* final state. (libc rand did produce different elongations on the same inputs -- seed1: [8,5,0,10,3], seed2: [8,5,0,8,2].) This is a real RNG x inputs coincidence, not a port bug. Adapted the test to use attempts_threshold inputs (10 ribosomes, 5 viable forward picks) where divergence is reliable. Documented in audit.md. Stub gate * tests/test_kinetic_charging_kernel_scaffold.py: renamed test_stubs_raise_not_implemented_until_filled_in -> test_no_stub_functions_remain. Iterates every documented kernel function and calls it with shape-minimal inputs; fails fast if any function still raises NotImplementedError. Task #2 (the full Cython kernel port) is now complete. Summary: The full upstream _trna_charging.pyx kernel (638 LOC Cython, 11 functions) is ported to v2ecoli/processes/polypeptide/kinetic_charging_kernel.py (~700 LOC Python+numba+docstrings). Parity is enforced by two committed goldens: - tests/fixtures/trna_charging_kernel_golden.json.gz (25 cases, libc-rand capture from upstream Cython at SHA 330ee3f4) - tests/fixtures/trna_charging_kernel_numpy_randomstate_golden.json.gz (10 stochastic cases, regenerated by capture_numpy_randomstate_golden.py) 8 deterministic functions assert byte-identity vs upstream. 3 stochastic functions assert byte-identity vs the per-RNG numpy golden, plus algorithmic invariants (conservation, non-negativity, convergence) vs the upstream libc golden. The kernel is ready to be wired into KineticTrnaChargingModel (Task #3). Results: pytest tests/test_kinetic_charging_kernel{,_scaffold}.py -> 24 passed, 0 skipped, 2.89s (under -m 'not sim') Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…caffold (Task 3a) New v2ecoli/processes/polypeptide/kinetic_charging.py defines KineticTrnaChargingPolypeptideElongation as a peer subclass of BasePolypeptideElongation -- matching v2ecoli's "one Process subclass per elongation model" convention (vs upstream's strategy pattern on a single PolypeptideElongation Process). config_schema dict-merges 12 new kinetic-charging-specific keys onto the base: codon_sequences, residue_weights_by_codon, n_codons, i_start_codon, is_map_substrate, n_trna_codon_pairs, trnas_to_codons, codons_to_amino_acids, k_cat__per_s, K_M_amino_acid__per_L, K_M_trna__per_L, reconciliation_buffer. Defaults are empty/zero-shaped; Task #5 populates from sim_data.relation when the composite is built. 14 method stubs (initialize, get_kinetic_constants, elongation_rate, request, run_model, codon_sequences_width, sequences, max_charging_rate, final_amino_acids, evolve, reconcile, protein_maturation, monomer_to_aa, monomer_limit) each raise NotImplementedError with an explicit task marker ("Task 3b" / "Task 3c" / "Task 3d" / "Task 3e") in the message. Catches any accidental composite build against the partial port -- the runtime error tells you which session to pick up next. Smoke test: tests/test_kinetic_charging_polypeptide_elongation_scaffold.py -- 20 tests, all green in 3.80s: * 6 structural checks (module imports, subclass relationship, name/topology inheritance, all 12 kinetic config_schema keys present, no base keys dropped by the dict-merge, schema entries well-formed with _type and _default). * 14 parametric checks (one per stubbed method) verifying the task-marker string appears in each stub's source. No circular import: polypeptide_elongation.py doesn't import the new file -- 3f's composite arch wrapper is the only consumer of the new class. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replaces 2 stubs in v2ecoli/processes/polypeptide/kinetic_charging.py. initialize(config) (~90 LOC) -- calls super().initialize(config) to lay down base attrs, then unpacks the 12 new kinetic-charging-specific config_schema keys: * Constants: cell_density (from base's cellDensity key). * Codon sequences: protein_sequences, monomer_weights_incorporated, n_monomers, i_start_codon, is_map_substrate. * Tools: n_trnas, n_codons + the 6-segment slice layout for the molecules buffer (slice_free_trnas / charged_trnas / amino_acids / charging_counter / reading_counter / codons_to_trnas_counter). * Mapping arrays: trnas_to_amino_acids, amino_acids_to_trnas, trnas_to_codons, codons_to_trnas, codons_to_amino_acids, plus the derived trnas_to_amino_acid_indexes (per-tRNA aa index, np.int8 for kernel memory footprint). * max_attempts = np.byte(4), buffer = reconciliation_buffer. * Kinetic params: k_cat__per_s, K_M_amino_acid__per_L, K_M_trna__per_L. * previous_rate warm-start for next tick's binary search. get_kinetic_constants(cell_mass) (~5 LOC) -- volume-scales the per-litre Michaelis constants back to per-cell. Notable upstream-vs-v2ecoli diffs documented in the docstrings: * self.process.X -> self.X (v2ecoli's class IS the process; no parent ref). * "cellDensity" matches the base schema key (upstream "cell_density"). * n_avogadro already set by base; not re-fetched. * pint Quantities throughout (.to(units.L).magnitude) instead of Unum. Tests ----- tests/test_kinetic_charging_polypeptide_elongation_scaffold.py grows from 20 to 23 tests: * Drops initialize and get_kinetic_constants from the marker-list (they're no longer stubs). * test_initialize_is_no_longer_a_stub -- fails loud if either regresses. * test_initialize_sets_documented_attrs_via_source_scan -- 27 expected self.X = ... assignments must appear in initialize's body. Pure source scan, no cache needed. * test_get_kinetic_constants_uses_cell_volume_conversion -- guards against regression to the pre-port "return inputs verbatim" form. * @pytest.mark.sim @_needs_cache: test_initialize_runs_end_to_end_against_cache -- loads real cache, augments with synthetic kinetic extensions via the new _make_kinetic_extensions helper, instantiates the class, verifies all attrs (n_trnas / n_codons / molecules_input_size / slice contiguity / trnas_to_amino_acid_indexes / max_attempts / buffer / previous_rate). * @pytest.mark.sim @_needs_cache: test_get_kinetic_constants_returns_volume_scaled_arrays -- doubling cell_mass doubles the returned K_M arrays. Results ------- pytest tests/test_kinetic_charging_polypeptide_elongation_scaffold.py -> 23 passed, 1 warning, 3.74s pytest tests/test_kinetic_charging_*.py -> 47 passed, 1 warning, 3.30s -- no regressions. The _make_kinetic_extensions helper is exported in the test module for 3c-3e to reuse without re-writing the synthetic-config boilerplate. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Six stubs ported in v2ecoli/processes/polypeptide/kinetic_charging.py plus
one base-method override.
_init_bulk_indices(bulk_ids) -- override extending base with atp_idx,
amp_idx, ppi_idx, met_idx, map_idx. Mirrors upstream's
PolypeptideElongation.calculate_request block at lines 534-538.
elongation_rate(states) -- re-derives (protein_indexes, peptide_lengths)
from states["active_ribosome"] (v2ecoli's contract); builds
self.longer_sequences (codon-based) via buildSequences; calls
kernel.get_elongation_rate; updates self.previous_rate warm-start.
request(states, aasInSequences) -- ignores aasInSequences (amino-acid-
domain, wrong for the kinetic model); recomputes codon-domain
monomers_in_sequences from self.longer_sequences; runs run_model against
"bulk_total"; builds bulk requests for AAs (+1% buffer), ATP, both tRNA
pools, synthetases, MAP, and water (incl. termination cleavage); returns
(fraction_charged, amino_acids_used.astype(float), requests).
run_model(codons, attr, states) (~270 LOC) -- drives a scipy.integrate.
solve_ivp RK45 ODE over the 6-segment molecules buffer (slice layout was
set up in initialize). Inner ode_model closure computes charging+reading
rates with sin/sin^2 roll-offs for low-charged-fraction and low-AA-pool
corner cases. On attr="bulk_total" precomputes K_M_amino_acids, K_M_trnas,
cell_amino_acid_saturation for the subsequent "bulk" call to reuse.
Discretizes outputs (ceil for sizing, stochasticRound for evolve), caps
at AA availability, reconciles tRNA-pool under/overflow. Emits
trna_charging.{saturation_trna, turnover} listener fields on "bulk".
max_charging_rate(states, attr) -- v_max = k_cat__per_s * n_synthetases.
Uses self.synthetase_idx (v2ecoli) for upstream's
self.process.trna_synthetases_for_aas_idx.
codon_sequences_width(elongation_rates) -- returns self.sequences_width
cached by elongation_rate.
sequences(sequences) -- returns self.longer_sequences. The sequences arg
is intentionally ignored (kept for API parity with upstream).
Upstream-vs-v2ecoli diffs documented in each docstring:
* self.process.X_idx -> self.X_idx (v2ecoli's class IS the process).
* Unum .asNumber(...) -> pint .to(...).magnitude.
* v2ecoli's request(states, aasInSequences) contract vs upstream's
(states, monomers_in_sequences, protein_indexes, peptide_lengths).
Tests
-----
tests/test_kinetic_charging_polypeptide_elongation_scaffold.py grows from
23 to 28 tests:
* Removes 6 entries from the test_method_stub_carries_task_marker
parametric list (ported now).
* test_3c_method_no_longer_stub (parametric, 7 methods incl.
_init_bulk_indices) -- fails loud if any reverts.
* test_elongation_rate_calls_kernel_and_sets_longer_sequences --
source-scan for kernel.get_elongation_rate, buildSequences,
longer_sequences, sequences_width, previous_rate.
* test_request_requests_all_kinetic_bulk_keys -- source-scan that all 7
bulk indices (amino_acid/atp/uncharged_trna/charged_trna/synthetase/
map/water) are referenced + v2ecoli return tuple is present.
* test_run_model_uses_ode_and_kernel_constants -- source-scan for
solve_ivp RK45 + rtol=1e-4 + atol=1e-7 + K_M_amino_acids/K_M_trnas +
stochasticRound + 7-tuple return.
* test_init_bulk_indices_adds_kinetic_keys -- source-scan that
super()._init_bulk_indices is called and the 5 new indices are referenced.
Source-scan was chosen over runtime because run_model end-to-end requires
both the cache to populate the kinetic config keys (currently synthetic
shape-only via _make_kinetic_extensions) AND the
active_ribosome/bulk/listeners state to be plausibly initialised. Full
runtime tests come together in Task 3f's behavior test after Task #5
wires up sim_data.
Results
-------
pytest tests/test_kinetic_charging_polypeptide_elongation_scaffold.py
-> 28 passed, 1 warning, 4.74s
pytest tests/test_kinetic_charging_*.py -m 'not sim'
-> 50 passed, 2 deselected, 1 warning, 3.52s -- no regressions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Scope expansion flagged: v2ecoli's BasePolypeptideElongation.evolve_state
is amino-acid-centric (polymerizes against aa_counts_for_translation). The
kinetic model needs codon-based polymerize -> reconcile -> protein_maturation
-> evolve, so we must override evolve_state entirely. evolve_state also
consumes monomer_to_aa, monomer_limit, next_amino_acids (originally slated
for 3e); pulled forward so the override is functional. 3e is collapsed into
3d.
evolve_state(timestep, states) (~225 LOC, override)
-- Codon-based replacement for the base evolve_state. Builds both AA sequences
(for polymerize) and codon sequences via monomer_limit. Polymerizes against
the codon limit. Calls reconcile -> protein_maturation -> evolve. Emits
the listener block (effective_elongation_rate, aa_count_in_sequence,
aa_counts, actual_elongations/_hist, did_terminate, termination_loss,
num_trpA_terminated, process_elongation_rate). Drops upstream's
non-kinetic branches (served by other v2ecoli classes).
reconcile(states, result) (~80 LOC) -- runs run_model against "bulk",
compares predicted vs realised per-codon usage, seeds kernel.seed, dispatches
through kernel.reconcile_via_ribosome_positions (first pass) and
kernel.reconcile_via_trna_pools (fallback). Emits initial_disagreements,
charging_events, reading_events, codons_to_trnas_counter listener fields.
protein_maturation(states, did_terminate, terminated_proteins,
protein_indexes) (~65 LOC) -- MAP kinetic capacity (k_cat=6/s, per-cell
concentration) caps cleavages. Deferred terminations rolled back via
multinomial sampling. Unum -> pint conversions throughout.
evolve(states, total_aa_counts, amino_acids_used, next_amino_acid_count,
n_elongations, n_initialized, net_charged, monomer_usages,
initial_methionines_cleaved) (~55 LOC, 9-arg signature) -- bulk deltas:
initialization water, net tRNA charging, AA used, ATP/AMP/PPi per net
charging event, proton per charged-tRNA-mediated incorporation, water per
direct elongation, water consumed + Met released per cleaved initial Met.
monomer_to_aa(monomer) -- codons_to_amino_acids @ monomer.
monomer_limit(states, _) -- returns (codons_kinetics_model,
codons_to_amino_acids @ codons_kinetics_model).
next_amino_acids(...) -- returns 0 (matches upstream Base default).
final_amino_acids(...) -- kept as NotImplementedError raise with an
explanatory bypass message ("kinetic model uses monomer_limit via the
evolve_state override").
Tests
-----
tests/test_kinetic_charging_polypeptide_elongation_scaffold.py grows from
28 to 31 tests:
* Drops the parametric test_method_stub_carries_task_marker (all entries
ported now).
* test_no_task_3_stub_markers_remain_in_any_method (sentinel: scans every
method's source for `raise NotImplementedError` paired with `Task 3X`;
docstring cross-refs ignored).
* test_evolve_state_is_overridden_with_codon_pipeline (override sanity +
9 kinetic-hook references).
* test_reconcile_calls_both_kernel_helpers.
* test_protein_maturation_uses_map_kinetics.
* test_evolve_emits_all_kinetic_bulk_deltas (all 9 bulk indices).
* test_final_amino_acids_raises_kinetic_bypass_message.
* test_monomer_to_aa_uses_codons_to_amino_acids_matmul.
* test_monomer_limit_returns_kinetic_prediction_tuple.
* test_next_amino_acids_returns_zero_placeholder.
Results
-------
pytest tests/test_kinetic_charging_polypeptide_elongation_scaffold.py
-> 31 passed, 1 warning, 2.85s
pytest tests/test_kinetic_charging_*.py -m 'not sim'
-> 53 passed, 2 deselected, 1 warning, 2.99s -- no regressions.
Full evolve_state end-to-end gates on Task #5 (sim_data populates the new
config_schema keys) + Task 3f's behavior test. Source-scan suffices for now.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…Task #5) LoadSimData.get_polypeptide_elongation_config splatters a new partial dict from _kinetic_charging_extensions(relation) at the end of the existing config, so the 12 kinetic-charging-specific keys land alongside the steady-state ones. KineticTrnaChargingPolypeptideElongation consumes them via its config_schema dict-merge from 3a; SteadyState/Supply ignore them. _kinetic_charging_extensions returns: * codon_sequences, residue_weights_by_codon, n_codons, i_start_codon, is_map_substrate (the codon-table inputs) * n_trna_codon_pairs, trnas_to_codons, codons_to_amino_acids (the tRNA<->codon mapping) * k_cat__per_s, K_M_amino_acid__per_L, K_M_trna__per_L (the kinetic parameter arrays, Unum -> pint via unum_to_pint -> .to(...).magnitude, Avogadro-scaled to per-litre molecule counts; default fallback values match upstream's selenocysteine-canonical 1e4/s / 1 umol/L) * reconciliation_buffer Soft-fail behavior: returns {} when sim_data has no relation OR when relation lacks codon_sequences (the 145-line pre-port stub state). Both mean the cache predates Task #6 + Task #8 (ParCa rerun). Soft-failing keeps the existing steady-state path working; composites selecting the kinetic process will fail loud at composite build when their config_schema reads come up empty/zero-shaped. Other library deltas evaluated and deferred (audit.md documents each): * initial_conditions.py +61 -- kinetic initialize. Revisit only if 3f's behavior test surfaces a divergence; steady-state init may be enough. * schema.py +65 -- upstream's strategy-pattern listener schema. v2ecoli's listener schema lives in v2ecoli/types/; no direct equivalent. * parquet_emitter.py +139 -- upstream refactor superseded by v2ecoli's feat/default-baseline-parquet PR. * logging_tools.py +9 / json_state.py +5 -- trivial / unrelated. Tests ----- tests/test_kinetic_charging_sim_data_config.py -- 8 tests: * 2 soft-fail tests (None / pre-port stub). * 4 mock-based behavior tests (all 12 keys present with right shapes/ dtypes, k_cat in 1/s, K_M_amino_acid scaled by Avogadro). * 2 source-scan tests (extension is splatted into the config; docstring documents the Unum -> pint boundary). Cache invalidation ------------------ v2ecoli/library/sim_data.py is in cache_version.INPUT_FILES, so this change auto-bumps the cache fingerprint. The existing load_cache_bundle reads pickles directly without verify_cache_version, so cache-gated 3b tests still work against the stale cache via the soft-fail path. Composite builds via build_composite will surface StaleCacheError until Task #9 rebuilds via scripts/build_cache.py. Results ------- pytest tests/test_kinetic_charging_sim_data_config.py -> 8 passed, 1 warning, 3.91s pytest tests/test_kinetic_charging_*.py tests/test_polypeptide_elongation*.py -m 'not sim' -> 62 passed, 8 deselected, 1 warning, 3.69s -- no regressions. pytest tests/test_kinetic_charging_polypeptide_elongation_scaffold.py -> 31 passed (incl. cache-gated 3b tests exercising the soft-fail path). Unblocks Task 3f's composite arch (the config keys now exist). The end-to-end behavior test waits on Task #8 (full ParCa rerun) to ship a sim_data carrying the post-port Relation attrs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… (Task 3f) New v2ecoli/composites/kinetic_charging_baseline.py (~115 LOC) registers the "kinetic_charging_baseline" architecture via @composite_generator. Sibling of baseline.py that swaps SteadyStatePolypeptideElongation for KineticTrnaChargingPolypeptideElongation; everything else (allocator wiring, listener emission, partition orchestration, emitter declarations) matches baseline tick-for-tick. Pragmatic delegation -------------------- millard_pdmp_baseline.py (the existing sibling architecture) is 832 LOC of mostly-duplicated orchestration. For 3f I chose the smaller path: 115 LOC of delegation + a scoped PARTITIONED_PROCESSES swap through a context manager, rather than 600+ LOC of copied orchestration. The swap is bounded by try/finally so the original SteadyState mapping is restored even on exception, blocking a dirty-mutation leak into subsequent baseline() builds in the same Python process. Documented as future-work-able in the module docstring: factor baseline._build_baseline_doc(..., partitioned_processes=PARTITIONED_PROCESSES) and make this module a single-line delegation. Deferred -- out-of-scope for this session. Registration ------------ * v2ecoli/composites/__init__.py -- appended to side-effect imports + __all__. Importing v2ecoli.composites fires the decorator and registers the architecture. * v2ecoli/library/cache_version.py -- added to INPUT_FILES so changes to this composite auto-invalidate the cache fingerprint. Tests ----- tests/test_behavior_kinetic_charging.py -- 9 tests: Smoke (7, all pass): * Module imports cleanly. * Composite function has the expected (core, *, seed, cache_dir, config_overrides, bundle) signature. * Architecture is registered (looked up by entry.name in _REGISTRY since registry keys are fully-qualified module IDs). * Context manager swaps PARTITIONED_PROCESSES on entry and restores on exit, including on exception. * New composite is in cache_version.INPUT_FILES. * composites.__init__.__all__ exports the new arch. Cache-gated end-to-end (2): * test_composite_builds_against_stale_cache_via_soft_fail (@SiM @_needs_cache, passes) -- even against the pre-port cache, the composite builds: kinetic config splat is empty (soft-fail from Task #5), so the Process's initialize sees empty/zero-shaped defaults. Walks to doc.state.agents.<id>.ecoli-polypeptide-elongation_requester to confirm Steps are wired. * test_composite_runs_one_tick_with_kinetic_elongation (@_post_port_cache, skipped) -- gated on whether the cache's sim_data.relation carries the new attrs (codon_sequences). Activates after Task #8 reruns ParCa. Results ------- pytest tests/test_behavior_kinetic_charging.py -> 8 passed, 1 skipped, 22 warnings, 5.02s pytest tests/test_kinetic_charging_*.py tests/test_behavior_kinetic_charging.py -m 'not sim' -> 68 passed, 4 deselected, 1 warning, 3.16s -- no regressions. Task #3 complete. The full upstream KineticTrnaChargingModel port lands as: * v2ecoli/processes/polypeptide/kinetic_charging.py (~975 LOC) -- 3a-3d * v2ecoli/composites/kinetic_charging_baseline.py (~115 LOC) -- 3f * v2ecoli/library/sim_data.py extension (~95 LOC) -- Task #5 * tests/test_kinetic_charging_*.py + test_behavior_kinetic_charging.py (~1200 LOC) -- all sub-tasks End-to-end behavior verification gates on Task #8 (full ParCa rerun). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…nd tick passes (Task #8) Full ParCa pipeline runs in 194s (10 CPUs); cache rebuilt; the one-tick kinetic-charging behavior test that was gated on a fresh sim_data now PASSES. ParCa fixture refresh --------------------- * models/parca/parca_state.pkl.gz: 39 MB -> 42 MB (carries the Task #6 Relation port's kinetic attrs: codon_sequences shape (4309, 2409), synthetase_to_k_cat / K_A / trna_to_K_T dicts, etc.) * models/parca/runtimes.json: 194.3 s total. Per-step: 5.1/0.1/12.5/79.0/ 70.3/12.1/0.0/0.2/7.5. Docs estimate 4-8 h reflects a much slower box. Pre-flight bugs in the ParCa pipeline (3, all data-flow gaps from Task #6's Relation port not fully wiring into v2ecoli's KB): 1. MoleculeIds.start_codon missing -- 1-line add to molecule_ids dict. Mirrors upstream trna_charging_final. 2. 3 proteins (EG11357-MONOMER, EG11413-MONOMER, MONOMER0-4391) have mRNA-vs-protein mismatches in v2ecoli's KB; Relation skips them in _codon_sequences, but _build_codon_based_translation indexed the dict unconditionally -> KeyError. Defensive .get(mid, [start_codon]) sentinel. 3. setInitialRnaExpression's ids_tRNA_cistrons uses cistron form (alaT-tRNA), but Task #6's Relation produces [c]-suffixed (alaT-tRNA[c]). Strip the [c] suffix when building the tRNA_id_to_dist lookup. Cache rebuild ------------- Per docs/generate_full_parca.md: * out/workflow/simData.cPickle via hydrate_sim_data_from_state. * out/cache/* via save_cache (initial_state.json 10 MB, sim_data_cache.dill 167 MB). ~15 s. End-to-end load_cache_bundle gives all 12 kinetic config keys with realistic shapes (codon_sequences (4309, 2409), trnas_to_codons (63, 86), k_cat__per_s (21,), etc.) -- Task #5's _kinetic_charging_extensions splat now produces a full dict instead of empty. Runtime bugs in the kinetic Process port (4, all caught by activating the previously-skipped one-tick behavior test): 1. get_kinetic_constants Unum->pint regression: v2ecoli passes cell_mass as a pint Quantity (vs upstream's plain float); cell_mass * units.fg then produces fg^2*L/g. Normalize with hasattr(cell_mass, "to") check. 2. max_charging_rate shape mismatch: v2ecoli synthetase_idx covers 22 enzymes; k_cat__per_s is per-AA (21). Project all-enzyme count onto AAs via aa_from_synthetase @ counts. 3. request returned per-tRNA fraction_charged (86) but v2ecoli base listener expects per-AA (21). Aggregate via aa_from_trna @ charged / aa_from_trna @ (free + charged) before normalizing. 4. protein_maturation strip-units called float() on a non-scalar. n_can_cleave_q.to(...).magnitude can be shape-(1,) ndarray. Use float(np.asarray(...).sum()). Behavior test path fix: walk to first_agent.unique.active_ribosome (the composite wraps unique molecules under state.agents.<id>.unique). Results ------- pytest tests/test_kinetic_charging_*.py tests/test_behavior_kinetic_charging.py -> 72 passed, 42 warnings, 10.94 s The kinetic composite builds against the fresh cache, runs one tick without crashing, and ribosome counts stay positive. Task #3f's end-to-end gate clears. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…xisting env failures) `pytest -m 'not sim and not slow'` -> 338 passed, 2 failed in 33.9s. Both failures (test_analysis_runner.py, test_workflow_smoke.py) are pre-existing on main -- ImportError on the missing parquet extra, not regressions from Task #8's ParCa rerun or the kinetic-charging port. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…(Task #11) pytest tests/test_model_behavior.py -> 7 passed, 23.07s. No threshold adjustments needed -- the new fixture is biologically valid: * dry-mass doubles within [1.5, 2.5] * replication completes within the [900, 1800] s window * division fires with bulk conservation * daughters are viable (build + grow) Pre-requisite: the first 4 tests need out/workflow/single_cell_meta.json (chromosome snapshots over a full cycle). Generated by running python reports/workflow_report.py. The trajectory came out clean -- division at t=2491s, dry_mass=702.8fg, 2 chromosomes, 11.9x realtime. Patches to reports/workflow_report.py Task #8's cache rebuild made listeners.mass.* values pint Quantities (femtogram-typed) instead of bare floats. ~27 call sites in workflow_report.py did float(mass.get('dry_mass', 0)) etc., which fails on Quantities with DimensionalityError. * Added _fg(value) helper at the top: detects hasattr(value, 'to') -> converts via .to(units.fg).magnitude; bare numbers pass through. * Bulk-replaced float(mass.get( -> _fg(mass.get( (~25 occurrences) and float(cell.get( -> _fg(cell.get( (~2 occurrences) via a sed pass. Same pattern as the Task #8 fix #4 in kinetic_charging.py. * One residual case (float(d_mass.get('dry_mass', 0)) in _run_daughter, Step 4) surfaces only in daughter simulations; deferred to Task #13's full-report cleanup. Step 2 of workflow_report produces enough output for all 7 behavior tests to run regardless. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…re refresh) scripts/parity_check.py --seconds 120 --compare ... --build-check ran both gates: * real-emitter build_composite: OK (the gate that catches emitter-schema resolve failures the null emitter hides). * null-emitter signature vs committed golden: structured drift, expected. Signature drift (vs tests/golden/baseline_parity_signature.json): bulk_sum Delta +0.30% cell_mass Delta +0.30% (1294.77 -> 1298.68 fg) dry_mass Delta +0.31% (388.63 -> 389.82 fg) listeners_n Delta +0.28% (91884 -> 92138) listeners_sum Delta +10% active_ribosome +170 (13542 -> 13712) active_RNAP +83 (869 -> 952) RNA +33 (3225 -> 3258) others unchanged or tiny All consistent with the Task #8 fixture refresh (new ParCa fixture replaces the old fast-mode one). No pathological deviation -- ratios and relationships preserved. Golden re-capture NOT done from this branch per AGENTS.md: Re-capture the golden (--out) only from a clean origin/main worktree when main's behavior intentionally changes. The trna_charging port PR's reviewer will see this audit and re-capture the golden from main after merge. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…orts
reports/workflow_report.py
* Three residual float(... .get('dry_mass', ...)) call sites that
survived the Task #11 bulk replace now use _fg(...) consistently.
* Open the main HTML output with encoding='utf-8' so the unicode chars
in the cytoscape diagrams + WorkflowVisualization markup don't trip
the default ASCII encoder.
v2ecoli/visualizations/_helpers.py
v2ecoli/processes/parca/viz/network.py
* Same UTF-8 fix in both write_outputs functions (json.write_text +
html.write_text). The ParCa composition diagram was silently skipped
before; now it renders.
Outputs (committed under reports/figures/trna-charging/ with -f since
reports/ is gitignored):
* trna_charging_session.html (153 KB) + timestamped archival copy --
pr_session_report.py before/after parity render: before captured
from /tmp/cache_main (main's parca_state.pkl.gz, fast-mode fixture),
after captured from the current trna_charging_final cache (Task #8's
full-mode fixture). 60 baseline steps each.
workflow_report.html and the cytoscape diagrams are produced under
out/workflow/ (gitignored at the workspace level); not committed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sixth UTF-8 site found this session: reports/multigeneration_report.py:418 open(args.out, "w") -> open(args.out, "w", encoding="utf-8"). The 3-generation simulation completed cleanly (gen 1: 224s wall, 381 -> 696fg, divided; gen 2: 247s, 351 -> 667fg, divided; gen 3: 245s, 336 -> 653fg, divided) but rendering crashed on the multiplication-sign / arrow chars in the visualization markup. audit.md updated with the full Task #13 progress log: 6 patched call sites, the pr_session_report.py before/after capture pipeline (extracting main's fixture to a parallel cache so the running multigen isn't disturbed), all-3-gens summary table. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Multigen re-run completed in 724s wall: 3 generations, all divided cleanly, multigeneration_report.html written (264 KB). UTF-8 patches held across all 4 render paths now. All blocking work for the kinetic-charging port is complete (Tasks #2-#13 except #4 which is independent of this branch). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…le (port complete) Surveyed all 10 process deltas upstream trna_charging_final introduced: metabolism.py +8 upstream use_trna_charging -> steady_state_* polypeptide_initiation +60 ribosome profiling listener protein_degradation +19 monomers_degraded listener transcript_elongation +30 test helper rewrite tf_binding +5 formatting chromosome_structure +58 ribosome profiling listener cell_division +22 Process -> Step cleanup monomer_counts +69 upstream feature removal ribosome_data +2 strict=False arg drop metabolism_redux_classic NEW file +130 (whole-new-arch territory) None are required for kinetic-charging end-to-end (the goal of this branch -- Tasks #2-#13 cover that completely). Per-file decisions: * All 9 incremental deltas: SKIP. Reasoning in audit.md table per file (refactor noise, cosmetic, new-feature without consumer, upstream regression-cleanup we shouldn't apply, etc.). * metabolism_redux_classic: DEFERRED. Real feature work -- needs a new composite arch + sim_data plumbing + behavior test, on the scale of the kinetic_charging_baseline arch from Task #3f. Belongs in its own PR, not bundled here. Task #4 close-out marks the entire trna_charging_final port complete: Tasks #1 #2(a-e) #3(a-f) #5 #6 #7 #8 #9 #10 #11 #12 #13 done. Task #4 closed-deferred with audit trail. session_transcript.md updated to reflect everything through this commit (1860 messages -> next refresh covers this commit too). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Merge remote main branch (619 commits ahead) into trna_charging_final. Resolves conflicts: - Composites __init__.py: combined imports with kinetic_charging_baseline - Binary pickle: used main version (parca_state.pkl.gz) - TSV file locations: git auto-resolved from flat/ to flat_overrides/ Maintains trna_charging model additions while incorporating dashboard pin bumps, registry fixes, and Docker/workflow updates from main. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
…ranch
Investigation kicking off the consensus elongation model build on a fresh
branch (forked from trna_charging_final). audit.md stratifies the three
target behaviors by ODE-coupling pattern: AA synthesis/import/export is a
real ODE-RHS extension (state vector 6 → 9 slices, supply terms added to
the AA balance), ppGpp is decoupled outer-loop work (port _ppgpp_request
/ _ppgpp_evolve from SteadyStatePolypeptideElongation), and aa_count_diff
becomes trivial once supply accumulators exist (one-substitution fix to
the latent {}-return bug at kinetic_charging.py:775). 5-phase plan with
P2 (ppGpp) and P3 (ODE merge) independent and parallelizable.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…r slices Adds the include_aa_supply boolean parameter (default False) to KineticTrnaChargingPolypeptideElongation and conditionally extends the ODE state-vector layout from 6 to 9 slices when on. The three new slices (slice_total_synthesis / _import / _export) are allocated but not yet written by the RHS — that's Phase 3b. With the flag off, the layout is bit-identical to trna_charging_final@5ffb76de, verified by 40 existing regression tests. 7 new scaffold tests in test_kinetic_aa_supply_ode_scaffold.py: - 4 schema/source introspection (no cache required) - 3 cache-backed initialize tests (flag-off bit-identical layout; flag-on 9-slice layout with contiguous accumulators; accumulators initialize to 0) - 1 phase-boundary marker that fails the moment P3b RHS writes land See workspace/investigations/consensus_elongation/audit.md §2 for scope. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ic initialize Adds six instance-attr unpacks to KineticTrnaChargingPolypeptideElongation's initialize() after super().initialize: amino_acid_synthesis, amino_acid_import, amino_acid_export, aa_supply_scaling, get_pathway_enzyme_counts_per_aa, and import_constraint_threshold. These are the callables/threshold that SteadyState unpacks at polypeptide_elongation.py:923-935 but BasePolypeptideElongation does not. The kinetic class needs them for Phase 3b-ii's run_model, which builds a supply_function closure and feeds it into the kinetic ODE RHS. No behavior change: the new attrs are not yet called from any code path. The flag-off layout stays bit-identical (verified by 40 existing regression tests still passing). The composite-level config_overrides plumbing for include_aa_supply is verified working via the new P3b-ii red gates. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds tests/test_kinetic_aa_supply_ode.py with 8 tests that go red on the
P3a/P3b-i scaffold (no RHS edits yet) and pass only after P3b-ii actually
extends the kinetic ODE's RHS with AA synthesis/import/export terms.
Three test tiers per audit.md §2:
§A — Joint-integration proof:
- test_supply_on_aa_pool_higher_than_supply_off_after_one_tick: with
identical initial state and timestep, AA pool delta under flag-on must
exceed flag-off (supply was added inside the integrate). Currently
fails with byte-identical deltas, proving the flag is a no-op on the
RHS today.
- test_supply_accumulators_emit_nonzero_after_solve: the three accumulator
slots must integrate to non-zero flux.
- Two source-marker tests pin that dx_dt[slice_total_*] writes appear
inside ode_model, and that supply_function is invoked there (not just
before/after the solve).
§B — Listener emission:
- aa_synthesis / aa_supply / aa_exchange_rates listeners shape match
n_amino_acids when flag on; absent or zero when flag off.
§C — aa_count_diff bug fix (latent {} return at kinetic_charging.py:775):
- Source marker that `return net_charged, {}, update` is gone.
- End-to-end: aa_count_diff is ndarray with shape (n_amino_acids,) and
sign convention matches SteadyState (positive = over-supplied).
Cache-backed tests gated by @pytest.mark.sim + _needs_cache + post-port
relation guard. Confirmed config_overrides successfully wires
include_aa_supply through to the process (the composite built clean
with the flag set; the no-op RHS produces byte-identical AA deltas).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The binding ODE merge from the design spec: amino acid synthesis, import,
and export are now integrated INSIDE the same RK45 solve as tRNA charging,
not applied as a separate pre/post step. ode_model's RHS calls a
get_charging_supply_function closure at every sub-step, evaluated at the
time-evolving aa_conc, and writes three accumulator dx_dt rows that
integrate the per-AA flux over the timestep.
Changes to v2ecoli/processes/polypeptide/kinetic_charging.py (+~280 LOC):
- _build_supply_function: per-tick helper that constructs the supply
closure. Mirrors SteadyState's _supply_args at polypeptide_elongation.py
:1056-1100 — pulls boundary external media, enzyme/importer/exporter
counts from bulk_total, computes the counts_to_uM_mag unit bridge, and
returns (supply_function, counts_to_uM_mag, supply_listeners).
- ode_model: signature extended with supply_function + counts_to_uM_mag.
When non-None, evaluates supply(aa_conc_uM) per RK45 sub-step, adds
+v_syn +v_imp -v_exp / counts_to_uM_mag to dx_dt[slice_amino_acids],
and writes the three accumulator dx_dt rows. AAs are now bidirectional
in the integrate, not a one-way drain.
- run_model: post-solve, extracts total_synthesis/import/export from the
accumulator slices, stores them on self._last_supply_totals for evolve()
to use, and emits aa_supply, aa_synthesis, aa_import, aa_export,
aa_exchange_rates listeners on the "bulk" pass.
- evolve: returns aa_count_diff = (synthesis + import - export) - aas_used
as ndarray[float64] when flag on; zeros[n_aas] when off. Fixes the
latent {}-return bug at trna_charging_final@5ffb76de:775.
- inputs/outputs overrides: add boundary port for external media reads;
declare the new AA-supply listener fields under growth_limits so
process-bigraph propagates them to the store.
- Latent bug fix (caught during P3b-ii testing): kinetic evolve_state was
deep_merging additional_listeners from reconcile() and protein_maturation()
at the wrong store path (top-level instead of nested under "listeners").
Pre-existing on trna_charging_final; surfaced only when P3b-ii tests
asserted on emitted growth_limits through the store. Fixed by wrapping:
deep_merge(update, {"listeners": additional_listeners}). Existing
trna_charging emissions (initial_disagreements, charging_events) were
also silently misrouted; now flow correctly.
P3b-ii test suite: 8/8 green. The binding ODE-merge proof
(test_supply_on_aa_pool_higher_than_supply_off_after_one_tick) confirms
the supply-on AA pool ends higher than supply-off after a 1-tick solve.
Listener emission + aa_count_diff formula equality both verified through
the store. 40 existing kinetic regression tests still pass — flag-off
path remains bit-identical.
P3a marker test flipped to assert RHS writes ARE present (boundary
between scaffold and semantics now crossed).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The kinetic_charging_baseline composite was already passing ppgpp_regulation=True via DEFAULT_FEATURES, but the kinetic class inherited only Base's ppGpp parameter scaffold — no _ppgpp_request / _ppgpp_evolve hooks — so ppGpp regulation was silently a no-op on the kinetic path. This commit wires it in: - initialize: unpack elong_rate_by_ppgpp + ppgpp_reaction_metabolites; build charging_params + ppgpp_params dicts that ppgpp_metabolite_changes consumes (mirroring SteadyState's setup at polypeptide_elongation.py :923-935). - outputs: declare rela_syn, spot_syn, spot_deg, spot_deg_inhibited under growth_limits so process-bigraph propagates the new fields to the store. - _ppgpp_request: ported verbatim from SteadyState (polypeptide_elongation .py:1481-1533). Computes ppGpp metabolite delta from the predicted post-charging tRNA split and returns the corresponding bulk requests. No-op when ppgpp_regulation=False. - _ppgpp_evolve: ported from SteadyState (polypeptide_elongation.py :1535-1632) with one adjustment — growth_limits listener fields use setdefault instead of replacing the dict, so the kinetic class's other writes to growth_limits (aas_used, aa_count_diff, net_charged) survive. No-op when ppgpp_regulation=False. Guards against the no-elongation edge case where aa_at_ribosome.sum() == 0 (f would be NaN). - request: computes counts_to_uM_mag (cached on self for _ppgpp_evolve), per-AA tRNA counts, ribosome/ppgpp/rela/spot concentrations, the per-AA sequence fraction f, and v_rib in μM/s, then appends the _ppgpp_request output to bulk requests. - evolve_state: calls _ppgpp_evolve after the existing bulk deltas are merged into update, so the ppGpp reaction sees the post-elongation tRNA state and is clamped by the partitioned ppGpp-reaction metabolite limits. - elongation_rate: when ppgpp_regulation=True and disable_ppgpp_elongation_inhibition is False, caps the binary-search target at elong_rate_by_ppgpp(ppgpp_conc, basal). This propagates through to target_codon_rate inside run_model — codons available to read this tick shrink in proportion to the ppGpp-driven slowdown. Test gates (tests/test_kinetic_ppgpp_coupling.py, 10 tests, all green): §A — Source markers (6, no cache): - _ppgpp_request / _ppgpp_evolve methods exist - request calls _ppgpp_request; evolve_state or evolve calls _ppgpp_evolve - elong_rate_by_ppgpp referenced in source - outputs declares the ppGpp listener fields §B — Listener emission + bulk delta (2, sim + cache): - rela_syn / spot_syn / spot_deg listeners non-empty with regulation=True - GUANOSINE-5DP-3DP[c] (ppGpp) bulk count actually changes after a 1-tick run — proves _ppgpp_evolve fired and applied deltas §C — Flag-off no-op (2, sim + cache): - Listener fields absent or zero with regulation=False - Composite runs cleanly with regulation=False P3 sim regression (6/6) confirmed unchanged after P2 edits. All 42 fast P2+P3+scaffold tests still green. The kinetic_charging_baseline composite running with ppgpp_regulation=True is now functionally the consensus elongation model — kinetic tRNA charging + ppGpp + AA-supply ODE merge all integrated. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Locks the consensus elongation model into a first-class artifact with
regression protection. Two pieces:
1. v2ecoli/composites/consensus_baseline.py — thin wrapper composite
that delegates to kinetic_charging_baseline with include_aa_supply=True
and ppgpp_regulation=True forced via a _CONSENSUS_DEFAULTS dict (user
overrides win on merge, so ablation experiments still work). Same
physics as kinetic_charging_baseline + flags — discoverability over
duplication. Registered in composites/__init__.py with a clean alias.
2. tests/test_consensus_composite.py — 9 tests in four layers:
§1 Smoke (5, no cache): module imports, composite registered under
both doubled and clean ids, signature parity with kinetic_charging_baseline,
_CONSENSUS_DEFAULTS forces both flags True, user overrides win on merge.
§2 Composite equivalence (1, sim): consensus_baseline() and
kinetic_charging_baseline(config_overrides={flags True}) produce
IDENTICAL first-tick AA deltas. Guards against the alias drifting
from the underlying composite.
§3 Cross-class numeric parity (1, sim): kinetic class's _ppgpp_request
and SteadyState's _ppgpp_request produce identical delta_metabolites
on synthetic inputs (same seed, same params). Locks in that P2's
port is bit-faithful to SteadyState — they share the underlying
ppgpp_metabolite_changes math, so the wrappers must agree.
§4 Flag-off legacy regression (1, sim): kinetic_charging_baseline
with BOTH flags False runs to completion, AA pool drains
monotonically (no supply replenishment), ppGpp listener stays at
zero (no leak from the consensus path). Proves the consensus flags
gate cleanly and the legacy kinetic path is unbroken.
5/5 fast + 4/4 sim tests green. P2 + P3 sim regression unchanged.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
scripts/consensus_validation_sweep.py is the multi-media × multi-seed × multi-generation orchestrator for the Phase 5 validation step from audit.md. Delivers the script + a thin metrics/report pipeline; the actual multi-hour sweep run is a user-driven follow-up. Pieces: - preflight: cache existence + post-port relation guard + media-id validity + composite-build check. Runs in <1 min via ``--preflight-only``. Surfaces stale-cache / missing-media / build errors BEFORE the multi-hour sweep launches. - build_run_specs / RunSpec: Cartesian product of media × seeds with resumability semantics (skip if db exists; --force to overwrite). - execute_run: builds consensus_baseline composite with media_id pinned via config_overrides; runs through run_multigen_sqlite with single_daughters=True (memory-bounded). Per-condition output to <out>/<media>/seed-<N>/run.db. - compute_metrics: reads a finished run.db and extracts the consensus metrics — growth rate (log-fit on cell_mass), mean tRNA charging fraction, ppGpp count statistics, mean elongation rate. NaN-safe; populates Metrics.notes for any field that can't be computed. - check_acceptance: per-criterion pass/fail against the spec's acceptance targets (charging ≥85%, ppGpp 2-5× rise on starvation, growth-rate plausibility). Reports actual + expected for each. - generate_report: JSON dump of per-run metrics + acceptance summary to <out>/report.json. Returns the dict for programmatic use. Spec deviation: the design spec's ``no_glucose`` condition isn't a registered v2ecoli media (it would need a boundary.external override at composite-build time). Substituted ``minimal_minus_oxygen`` as a starvation-class condition — also triggers the stringent response per biology, also exercises the ppGpp pathway. Easily overridden via ``--media``. tests/test_consensus_validation_sweep.py — 16 unit tests covering run-spec construction, pre-flight against missing + real caches, metric extraction from synthesized sqlite history rows, exponential fit numerics, acceptance evaluation, report round-trip, CLI arg parsing, and main()'s preflight-only short-circuit. All green; runs in 2 sec (no multi-hour sims). Preflight smoke-tested against the real cache: 4 default media valid, consensus_baseline builds clean, exit 0. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Profile of one consensus tick showed RK45 driving ~1.9M ode_model calls per simulated second (~5 min wall) — the supply terms make the merged ODE numerically stiff and RK45's stability region collapses to micro- steps. SteadyState's analogous merged ODE uses BDF for the same reason (see kinetics.py:404). One-line fix: pick BDF when supply_function is non-None, RK45 otherwise. Preserves the legacy bit-identical path when include_aa_supply=False. Re-profile result: ode_model calls per tick: 1,926,908 → 178,354 (10.8× fewer) per-tick wall: 304.06s → 41.92s (7.3× faster) 47/47 fast tests still green (including consensus + parity gates) Per-call ode_model cost is essentially unchanged (~140μs vs ~116μs); the supply closure itself is cheap. The slowness was entirely in RK45's adaptive step controller failing on the stiff system. Updated test_run_model_uses_ode_and_kernel_constants to assert both RK45 and BDF appear in source (the integrator is now chosen conditionally). Further win available: porting SteadyState's jac_sparsity hint (kinetics.py:393-405) would let BDF perturb fewer state columns per Jacobian update — could shave another 2-3×. Deferred until validation data tells us whether the current ~42s/tick is the binding constraint. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…loop Two bugs surfaced by the Stage A smoke sweep (the first end-to-end multi-tick run of the consensus model): 1. Unit conversion bug at the supply boundary ---------------------------------------------- amino_acid_synthesis/_import/_export at parca/.../metabolism.py:2001- 2119 expect aa_conc in METABOLITE_CONCENTRATION_UNITS (= mol/L). ode_model was passing μM magnitudes — interpreted as 1e6× too high. At ~1M apparent AA concentration, the synthesis function returns large reverse fluxes (heavy degradation), the AA pool drains, tRNAs become uncharged, RelA fires, and the cell spuriously starves at startup. Stage A smoke caught this: aa_supply listener = -143,760 counts/tick (negative — net AA destruction in minimal media at startup), fraction_trna_charged = 0.326 (vs 0.85 spec target), composite stopped at tick 10 with bare AssertionError. Fix: multiply aa_conc by unit_conversion (= 1e-6) at the supply call site in ode_model. Mirrors SteadyState's dcdt at kinetics.py:351 exactly. unit_conversion was already wired via self.charging_params["unit_conversion"] from the P2 init. 2. Pre-existing kernel: tRNA-recovery loop can drive chargings negative ---------------------------------------------------------------------- After fix #1, the consensus model has realistic charging activity and the AA-availability recovery loop at kinetic_charging.py:1745- 1749 started getting busy. The loop picks the most-over-rounded tRNA to decrement but never checks chargings > 0; under realistic load it picks already-zero tRNAs and drives them to -1, tripping the np.all(chargings >= 0) assert below. This bug exists in the legacy kinetic path too but only manifests under sustained heavy charging activity — the consensus supply terms make it routine. Fix: mask the argsort to only consider tRNAs with chargings > 0. If no charging events remain to undo for an AA, break the inner loop and accept the residual over-estimate. For the bulk_total pass that just makes the request envelope marginally looser (the allocator caps anyway); for the bulk pass it's logged but not fatal. The assert_message now explains the contract. Also includes: - scripts/consensus_validation_sweep.py: --stage flag (smoke / cell-cycle / cross-media / full), trajectory.json + summary.txt per-run with UTF-8 encoding fix, agents-store path unwrap in extract_trajectory / compute_metrics. - scripts/diagnose_consensus_tick.py: tick-by-tick runner without exception swallowing — produces a real traceback when run_model asserts fire. Used to localize bug #2. Verified via diagnose_consensus_tick.py 15 — 15 ticks complete cleanly with the fixes in place. 47/47 fast tests still pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Saves the full investigation report from the 2026-06-27 Stage A smoke sweep so the work can be picked up cleanly in a new session. New: workspace/investigations/consensus_elongation/stage_a_findings.md TL;DR: model functional, 100 ticks clean, charging 0.67 vs spec 0.85. Sections: reproduce instructions, quantitative results (per-tick wall 3.3s — order-of-magnitude better than the pre-Stage-A profile), full per-emit trajectory table, acceptance verdict, bug history with commits, four open biology issues with hypotheses and investigation paths, resume options (A: Stage B, B: investigate biology, C: close out), all artifact pointers. Updated: HANDOFF.md Phase status table now lists all 10 commits in chronological order with current status. Three next-session prompts (Stage B / biology investigation / close-out) ready to paste. Pointer block updated with current line numbers for all relevant kinetic_charging.py methods (initialize, request, evolve_state, evolve, _ppgpp_request, _ppgpp_evolve, _build_supply_function, run_model, ode_model, solve_ivp call, AA-recovery loop). Updated: audit.md Header status reflects "All phases landed, Stage A clean, biology partially-stressed". §6 phase table extended with the post-P5 work: BDF switch, unit + recovery fixes, Stage A, Stages B/C status. Also tracks: scripts/profile_consensus_tick.py — the A/B profiler used to localize the BDF stiffness problem. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Merged origin/main (231 commits) into consensus_elongation. The merge was clean (no textual conflicts); main's composite-registry infra and the feature's consensus/kinetic generators integrated automatically. Two fixes were needed for the CI suites to pass on the merged tree: - Add committed composite-state artifacts for the feature's two new generators (consensus_baseline, kinetic_charging_baseline), in both the clean-alias and doubled id forms. main added test_composite_state_coverage (every v2ecoli generator must ship a reports/composite-state/<id>.json); the two new generators lacked them. resolve_composite reads committed artifacts only (never live-builds), so the regen script can't bootstrap a brand-new generator — built directly from the generator doc's real trimmed state (agents/global_time). - test_write_summary_txt_is_readable: read the summary back with encoding="utf-8". write_summary_txt commits to utf-8 (emits a ≥), but the test used out.read_text() which defaults to the platform encoding (ascii under the CI locale) → UnicodeDecodeError. Encoding mismatch, now aligned with the writer's contract. Both CI suites green: fast 908 passed / 2 failures fixed; behavior 70 passed, 17 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.