Retry Bioconductor R-package installs to survive transient SSL blips - #196
Open
GuyTeichman wants to merge 1 commit into
Open
Retry Bioconductor R-package installs to survive transient SSL blips#196GuyTeichman wants to merge 1 commit into
GuyTeichman wants to merge 1 commit into
Conversation
CI job build (macos-latest, 3.12) flaked in
tests/test_differential_expression.py::TestDESeqRunner::test_run_deseq2_analysis
with a confusing "could not find function DESeqDataSetFromMatrix" error.
Root cause: BiocManager::install("DESeq2", ...) in deseq2_install.R downloads
~30 transitive Bioconductor/CRAN dependencies via download.file(), with no
retry. A single transient SSL connect error on one dependency (Biobase) only
emits a warning -- it does not make BiocManager::install() raise -- so the
install script still exits 0 with DESeq2 partially installed. The next R
script (deseq2_run_parametric.R) does `require("DESeq2")`, which silently
returns FALSE instead of halting, so the script proceeds and only fails much
later at DESeqDataSetFromMatrix() with a misleading "could not find function"
error surfaced to Python as ChildProcessError (rnalysis/utils/io.py).
Fix deseq2_install.R, limma_install.R, and rsubread_install.R to retry the
BiocManager::install() call up to 5 times with a 15s backoff (mirroring the
dl() retry wrapper already used for kallisto/bowtie2 downloads in
build_ci.yml), verifying via require(pkg, quietly=TRUE) that the package
actually loads before declaring success. Also raise R's default download
timeout to 300s. This lives in the R templates (not just CI config) so real
users hitting the same flaky-network install path benefit too, with no
change to installed package versions or to installs.py's existing
user-facing error messages.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016f1K2LQyVMdofggdMAEp9y
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
build (macos-latest, 3.12)(run 30758840006, job 91525659929) flaked ontests/test_differential_expression.py::TestDESeqRunner::test_run_deseq2_analysiswithChildProcessError: ... could not find function "DESeqDataSetFromMatrix".BiocManager::install("DESeq2", ...)indeseq2_install.Rdownloads ~30 transitive Bioconductor/CRAN dependencies with no retry. A single transient SSL connect error on one dependency (Biobase, downloaded frombioconductor.org) only produces an R warning, not an error, soBiocManager::install()does not raise and the install script still exits 0 with DESeq2 partially installed. The next script (deseq2_run_parametric.R) doesrequire("DESeq2"), which silently returnsFALSEinstead of halting, so execution proceeds and only fails much later atDESeqDataSetFromMatrix()with a misleading "could not find function" error, surfaced to Python asChildProcessErrorinrnalysis/utils/io.py.deseq2_install.R,limma_install.R, andrsubread_install.Rnow retry theBiocManager::install()call up to 5 times with a 15s backoff (mirroring the existingdl()retry wrapper used for the kallisto/bowtie2 downloads inbuild_ci.yml), verifying viarequire(pkg, quietly = TRUE)that the package actually loads before declaring success. Also raises R's default download timeout (60s -> 300s) for slow/degraded connections.installs.py's existing user-facing error messages/exception types, or to any serialized format.Test plan
conda run --no-capture-output -n rnalysis python -m pytest tests/test_differential_expression.py -k "not test_run_deseq2_analysis and not test_run_limma_analysis"— 22 passed (golden-file script-generation tests untouched by this change).conda run --no-capture-output -n rnalysis python -m pytest tests/test_installs.py -k "not test_install_limma and not test_install_deseq2 and not test_install_rsubread"— 13 passed.pytest --collect-onlyon both modules — no collection errors.Rscriptexecution of the modified install templates, and the realtest_run_deseq2_analysis/test_install_deseq2/test_install_limma/test_install_rsubreadintegration tests, could not be run. These will only be exercised for real by CI. I could not reproduce the transient SSL failure to confirm the retry fires in practice; the fix is based on careful reading of the BiocManager/R warning-vs-error semantics observed in the failing job's captured log.🤖 Generated with Claude Code
https://claude.ai/code/session_016f1K2LQyVMdofggdMAEp9y