Parallelize chunked matrix building approach (#818)#821

Merged

MaxGhenis merged 12 commits intomainfrom

maria/parallelizing

May 7, 2026

Commits on May 7, 2026

Extract ChunkedMatrixAssembler and stream CSR assembly (#818)

authored and
MaxGhenis
committed
Parallelize chunked matrix build across Modal workers (#818)

authored and
MaxGhenis
committed
Thread parallel-matrix flags through pipeline.py + workflow_dispatch

authored and
MaxGhenis
committed
Register build_matrix_chunk_worker via pipeline.py import

authored and
MaxGhenis
committed
Fix Modal app name used by dispatch_chunks_modal lookup (#818)

authored and
MaxGhenis
committed
Fix diagnostics upload script indentation

juaristi22
authored and
MaxGhenis
committed
Force Modal worker smoke lookup

juaristi22
authored and
MaxGhenis
committed
Allow Modal smoke test to use profile auth

juaristi22
authored and
MaxGhenis
committed
Set volume chunk dir for Modal matrix builds

juaristi22
authored and
MaxGhenis
committed
Release volume handles before Modal matrix reload

juaristi22
authored and
MaxGhenis
committed
Document Modal matrix runtime fixes

juaristi22
authored and
MaxGhenis
committed
Harden parallel matrix Modal dispatch
MaxGhenis
committed