Add conformal prediction notebooks - #185
Merged
victoraalves merged 5 commits intoJun 24, 2026
Merged
Conversation
Adds two notebooks under docs/notebooks/conformal/regression/: - 01_conformal_tutorial.ipynb: introduces split conformal prediction and adaptive (normalized) conformal intervals on a heteroscedastic regression dataset - 02_uncertainty_integration.ipynb: embeds conformal predictive sets into a Pyomo/OMLT optimization model with finite-sample feasibility guarantees, following the C-MICL framework Also: - Moves heteroscedastic_data.csv into the shared docs/notebooks/data/ folder - Adds notebook tests to test_run_notebooks.py - Adds citation for ovalle2026conformal to README.rst
Contributor
Author
|
Tagging @victoraalves and @carldlaird for review when you get a chance. |
…ification Adds two new classification notebooks under docs/notebooks/conformal/classification/ (01_conformal_tutorial.ipynb, 02_uncertainty_integration.ipynb) mirroring the existing regression series, and a third regression notebook (03_feasibility_verification.ipynb) that empirically illustrates the paper's ground-truth feasibility theorem by comparing standard MICL against C-MICL on a membrane reactor design problem, verified against a physics-based ODE oracle. Also fixes a stray typo in the regression uncertainty-integration notebook, adds the MNIST classifier weights and membrane reactor dataset used by the new notebooks, registers pytest coverage for all new notebooks, and adds ERA001/PLR0913 to the docs/notebooks ruff ignore list to accommodate illustrative commented-out code and the scipy.odeint calling convention.
Fixes the two ruff violations that previously required new per-file-ignore entries in pyproject.toml, instead of changing shared project config: - classification/01_conformal_tutorial.ipynb: removes the two illustrative commented-out retrain/save lines (ERA001) and updates the surrounding markdown to describe how to retrain without leaving dead code in the cell. - regression/03_feasibility_verification.ipynb: refactors membrane_reactor_odes to take a single packed parameter tuple instead of five separate scalar arguments (PLR0913), matching scipy.odeint's calling convention without exceeding ruff's argument-count limit. Both notebooks re-executed end-to-end with identical results, and pyproject.toml is now byte-for-byte unchanged from main.
victoraalves
self-requested a review
June 22, 2026 19:12
victoraalves
requested changes
Jun 22, 2026
victoraalves
left a comment
victoraalves
left a comment
Collaborator
There was a problem hiding this comment.
LGTM, just needs some cleanup regarding a few warnings from PyTorch/CUDA.
Addresses review feedback that the notebooks emit noisy warnings on import and during training/solving: - regression/01, regression/02, regression/03: TensorFlow/CUDA emits a plugin-registration banner (cuDNN/cuBLAS/cuFFT "already registered", computation placer warnings) at C++ static-initializer time, which predates any Python-level log-level setting. Silenced by redirecting the OS-level stderr file descriptor around the Keras import, plus a one-time throwaway fit() inside that same suppressed block to pre-trigger Keras's lazy CUDA-device probe (which otherwise re-emits the same banner on the first real fit/predict call in a later cell). - classification/02_uncertainty_integration.ipynb: importing from omlt.io.onnx transitively imports omlt's Keras reader (since Keras is installed), triggering the same banner even though this notebook never uses Keras directly. Wrapped the OMLT import in the same suppression helper. Also scoped a DeprecationWarning filter around the single torch.onnx.export call: the legacy TorchScript exporter (dynamo=False) is kept intentionally, since the newer torch.export-based exporter does not yet support dynamic_axes cleanly. - regression/03_feasibility_verification.ipynb: also silences a benign Pyomo W1002 warning, logged every time OMLT's Big-M formulation initializes surrogate variables to 0 before the solver assigns real values (the computed bounds never include 0, so this is expected). All 5 notebooks re-executed end-to-end with identical results to before, and pyproject.toml remains untouched (no shared lint config changes).
victoraalves
approved these changes
Jun 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds five Jupyter notebooks demonstrating conformal prediction and Conformal Mixed-Integer Constraint Learning (C-MICL), for both regression and classification:
docs/notebooks/conformal/regression/01_conformal_tutorial.ipynb: split conformal prediction and adaptive (normalized) intervals on a heteroscedastic datasetdocs/notebooks/conformal/regression/02_uncertainty_integration.ipynb: embedding conformal predictive sets into a Pyomo/OMLT optimization model, following the C-MICL framework (Ovalle et al., NeurIPS 2026)docs/notebooks/conformal/regression/03_feasibility_verification.ipynb: empirically illustrates the paper's ground-truth feasibility theorem by comparing standard MICL against C-MICL on a membrane reactor design problem, verified against a physics-based ODE oracledocs/notebooks/conformal/classification/01_conformal_tutorial.ipynb: conformal prediction sets for classification on MNIST, contrasting the traditional softmax-based nonconformity score with a logit-based score that avoids extra nonlinearities when embedded into an optimization formulationdocs/notebooks/conformal/classification/02_uncertainty_integration.ipynb: embeds the logit-based conformal classifier into a Pyomo/OMLT MILP to find minimal-perturbation adversarial examples under both standard and uncertainty-aware (C-MICL) formulationsAlso:
heteroscedastic_data.csvand addsmembrane_reactor_data.csvandmnist_classifier_500.pthto the shareddocs/notebooks/data/foldertests/notebooks/test_run_notebooks.pyovalle2026conformaltoREADME.rstafter the Zhang et al. entryERA001/PLR0913to thedocs/notebooks/*ruff ignore list to accommodate illustrative commented-out code and thescipy.odeintcalling conventionTest plan
jupyter nbconvert --execute)ruff checkandruff format --checkpass ondocs/notebooks/conformal/typospasses on changed filesLegal Acknowledgement
By contributing to this software project, I agree my contributions are
submitted under the BSD license.
I represent I am authorized to make the contributions and grant the
license.
If my employer has rights to intellectual property that includes these
contributions,
I represent that I have received permission to make contributions and
grant the required license on behalf of that employer.