Merge demux across flowcells/according to sample sheet manifests

[billytcl]

New issue checks

• I did not find an existing feature request.

Dorado subcommand

Demux

Feature request

We often do barcoded experiments across multiple flowcells. When we basecall we use the --recursive feature and feed in all the pod5s, which outputs a single bam file. However, during the demux process it will split them up by their respective flowcell IDs. The sample sheet feature also forces us to enumerate a unique flowcell ID for every output sample name. It ends up being a complete waste of time having to use samtools merge at the end to merge all the demuxed data back. It would be nice if the sample sheet lets us specify exactly what the output files should be.

[malton-ont]

Related to #1563

[weishwu]

Is my understanding correct that since v1.2.0, demux does not pool the bams from different flow cell folders (such as 20250603_1201_3E_PAA12345_e05460c6), even though they are just different washes from the same flow cell?

[malton-ont]

@weishwu,

Output file structure conforms to the specification found at https://software-docs.nanoporetech.com/output-specifications/26.01/minknow/output_structure/ and https://software-docs.nanoporetech.com/output-specifications/26.01/read_formats/bam/ - I believe different washes are considered separate acquisitions, so they will have a different start time (I'm not clear on whether the protocol_id is also different?) which will lead to a different file path.

[billytcl]

Is my understanding correct that since v1.2.0, demux does not pool the bams from different flow cell folders (such as 20250603_1201_3E_PAA12345_e05460c6), even though they are just different washes from the same flow cell?

Yes, it will be a different output folder, which is also part of the frustration. IMO if it's to conform to MinKNOW spec, the MinKNOW spec has to change....