Conversation
Add multi-threading support for FSL commands `topup` and `eddy` in the `dwifslpreproc` script, allowing efficient use of multiple CPU cores when the installed FSL version provides the `--nthr` option. A new `supports_nthr()` function queries FSL executables' help text to detect `--nthr` availability, accounting for version and hardware differences (notably, CUDA `eddy` lacks this option). The `nthr_option()` function constructs the appropriate `--nthr` value based on the script's thread request: defaults to all available system threads if unspecified, disables multi-threading for 0 or 1 threads, or uses the explicitly requested number. User-provided `--nthr` options in `-topup_options` and `-eddy_options` take absolute precedence, ensuring explicit configuration is never overridden. Prompt: > This worktree is based on the "master" branch of MRtrix3, which possess a different filesystem structure and build system to that indicated in the Claude project-level configuration. > The goal of this session is to modify Python command dwifslpreproc to make use of the multi-threading capabilities of the FSL software commands "topup" and "eddy". Engagement of multi-threading capability is dependent upon the particular version of FSL installed on the user system, which is outside of developer control; it therefore necessitates modulating behaviour based on probing that software for its capabilities. > In MRtrix3 Python module "run", a user-requested maximal number of threads (which may be specified from multiple sources) is registered in the Shared class. > Any requisite operations that are common to both "topup" and "eddy" commands should be functionalised in MRtrix3 Python module "fsl". > 1. For command "topup", newer versions of the software offer command-line option "--nthr". The presence of this command-line option in the installed FSL version must be confirmed by querying the command help page. If available, it should be added to the command invocation with an appropriate value. If run.Shared.get_num_threads() is 0 or 1, topup should be run with --nthr=1. If run.Shared.get_num_threads() is greater than one, then that value should be provided to the --nthr option. If run.Shared.get_num_threads() is None, meaning that there is no explicit user instruction regarding multi-threading, then the expected default behaviour for MRtrix3 is that all available threads be utilised; therefore the --nthr option should be specified, with the provided value equal to the number of threads available on the system as determined through the appropriate Python module. > 2. For command "eddy", the presence of the --nthr option will depend not only on the version of FSL installed, but also whether it is a CPU or CUDA GPU version of the "eddy" command being invoked. If it is the CPU version being executed, and the --nthr option is present in the help page, then utilise the same logic as presented in point 1 to augment the invocation with the --nthr option. > In both cases, first check the contents of dwifslpreproc user-specified options -topup_options and -eddy_options. If the relevant option is specified by the user, and already includes passing the --nthr option through to the relevant FSL command, then do not apply any of the logic above; the explicit user-specified content takes priority. Generated-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2254 (about 5 years late).
Turns out that for earlier versions of FSL,
eddychecked and obeyedOMP_NUM_THREADS, so I set that envvar within a custom environment variable dict within therunmodule, but then in a subsequent update they started ignoring that option and instead require explicit use of--nthroption. It's therefore necessary to query the version of FSL being invoked to discover whether that option is present.The same option has appeared in
topupto enable multi-threading. Though they don't align exactly in terms of FSL software version updates, so each has to be queried independently. I don't know exactly what the multi-threading performance is like, so safest to just pass through the user request / max number and lettopupdecide what to do.Ensures that if the user has included "
--nthr" in their option todwifslpreproc -topup_optionsor-eddy_options, this auto-fill doesn't happen.