Refactor scan grouping - #1061
Draft
tsalo wants to merge 48 commits into
Draft
Conversation
…into group-dwi-scans
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A phase2 file is always a sibling of phase1, never an estimation primary. Removing it from _GRE_SUFFIXES prevents a degenerate phase2-only source set from inferring PHASEDIFF and then raising StopIteration in to_fieldmap_info. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Locks the cross-session split behavior of named estimation groups (pepolar01 spanning two sessions -> pepolar01_ses-01 / pepolar01_ses-02) which no existing fixture exercised. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reproduce the legacy _split_named_member_groups_by_session behavior: a named (B0FieldIdentifier / IntendedFor) estimation group that spans multiple sessions is split into per-session estimators (e.g. pepolar01 -> pepolar01_ses-01 / pepolar01_ses-02), and application is derived independently from B0FieldSource. Locked by the dset_b0field_multisession golden fixture and a find_estimators unit test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…auto_id order Review fixes for find_estimators: - Sort the PE-axis and concatenation-group sets in ValueError messages so the golden snapshots are deterministic (set repr ordering was flaky run-to-run). - _intendedfor_buckets returns buckets in first-appearance (sorted-fmap) order, matching the legacy auto_NNNNN numbering instead of sorted-target order. - Drop a source-less IntendedFor session partition instead of raising 'Insufficient sources' (degenerate cross-session intent). - Add dset_intendedfor_autoid_order fixture locking the auto_id ordering. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wire find_estimators into group_dwi_scans via the new ordering: - _final_distortion_groups folds the estimator id into the physical signature, so a distortion group never spans two estimators (replaces build-then-refine). - _serialize_estimation_groups produces the legacy fmap_estimation_groups dict; application is derived by build_fmap_application_groups (preserving the exact many-to-many B0FieldSource mapping a one-to-one map could not represent). - IntendedFor estimators include their target series so the estimation group records the target dg ids, matching legacy _build_intendedfor_groups. - FieldmapEstimation gains an explicit bids_id param so a source carrying several B0FieldIdentifiers (belonging to several groups) is not flagged as a conflict; the axis check reads PE directly instead of building an estimation. - _as_list unpacks tuples (DwiSeries normalizes list identifiers to tuples) and distortion_signature normalizes a scalar B0FieldIdentifier to a 1-tuple, matching the legacy tuple(_ensure_list(...)) key. All 224 golden snapshots stay green and deterministic. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Code review found a behavior divergence: legacy collapses DWI runs sharing a physical signature into one distortion group BEFORE resolving field-map links, so per-run fmaps merge; the series-level estimator-fold split them instead. - _final_distortion_groups groups by physical signature alone (legacy build_distortion_groups). The legacy refine step is a no-op in practice (same-signature series share B0FieldIdentifier and resolve IntendedFor to the same group), so the estimator fold was wrong and is removed. - Distortion groups are now built before find_estimators, which takes them and resolves IntendedFor against whole distortion groups (dg-level bucketing), collapsing per-run fmaps into one estimation group exactly as legacy did. - find_estimators drops the vestigial series_to_estimator return; application is handled wholly by build_fmap_application_groups. - Add dset_same_sig_b0source and dset_same_sig_intendedfor fixtures + snapshots, verified byte-identical to legacy across all 8 flag combos each. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
No tests referenced the deleted internal helpers (all use the public API), so the only port needed was the pre-existing drbuddi failure: opposite-PE DWI runs form separate distortion groups and are not collapsed via an rpe_series field map. This fails identically on the pre-refactor baseline; marked xfail to document the desired behavior without falsely asserting it works. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Delete three now-unreachable functions ruff cannot detect: - _get_fmap_files (superseded by _subject_fmap_files) - _get_distortion_group_multipart_key (its only caller was deleted) - _group_by_sessions (pre-existing orphan; sole user of get_bids_params) and drop the now-unused get_bids_params import. Verified by repo-wide grep (no source or test references) and the full grouping/golden/fieldmaps suite. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Keeping as a draft for now.
Open issues
Summary
Reorganizes QSIPrep's scan-grouping and field-map-selection code into SDCFlows-style value objects, without changing behavior. The previously monolithic qsiprep/utils/grouping.py is split into a clear pipeline and shrinks from 1725 → 1055 lines.
PE-heuristic) → _serialize_estimation_groups + build_fmap_application_groups, with group_dwi_scans reduced to a thin orchestrator.
Why it's safe (behavior preservation)
group-dwi-scans baseline, covering every fixture × combine_scans/ignore_fieldmaps/estimate_per_axis combo, including error cases. The new code reproduces it exactly and deterministically.
B0FieldSource/IntendedFor), the four-dict output was verified byte-identical to the pre-refactor baseline across all 8
flag combos, locked by two dedicated fixtures.
Test plan
pre-existing/unrelated).
preserved fieldmap_info drives the same workflows.
Notes