Skip to content

Add test suite for backwards compatibility#999

Open
Jan Tilly (jtilly) wants to merge 2 commits intomainfrom
backwards-compatibility
Open

Add test suite for backwards compatibility#999
Jan Tilly (jtilly) wants to merge 2 commits intomainfrom
backwards-compatibility

Conversation

@jtilly
Copy link
Copy Markdown
Member

I'm adding a test suite to ensure that we remain backwards compatible in the sense that we can still predict from models that were fitted using an old version of glum.

Fits a GLM with each released minor version of glum (discovered dynamically via pixi search), pickles the model, then unpickles it under the current version and checks that predictions match.

Three new files under tests/backwards_compatibility/:

  • fit.py — fits and saves a model + predictions; accepts a version label or HEAD
  • run.py — orchestrates fit.py across all versions and compares predictions against HEAD

Runs on PRs via a GitHub Actions workflow. Also available as pixi run test-backwards-compatibility.

@MarcAntoineSchmidtQC
Copy link
Copy Markdown
Member

I like what this PR is doing, but I wonder if we need to create a dozen pixi environments every time. My understanding is that we want to ensure we are able to predict from an old model. Why not fit once and store the pickle file in git, then check in the current environment if we are able to load it.

@MarcAntoineSchmidtQC
Copy link
Copy Markdown
Member

Also, FYI for others that wants to test this locally. Older versions of pixi won't work. You need to update pixi (I had 0.63 and it didn't work).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants