Skip to content

Update build pipeline to use uv#7394

Open
mbollmann wants to merge 25 commits intomasterfrom
update-build-pipeline
Open

Update build pipeline to use uv#7394
mbollmann wants to merge 25 commits intomasterfrom
update-build-pipeline

Conversation

@mbollmann
Copy link
Member

@mbollmann mbollmann commented Feb 1, 2026

Resolves #7258, see there for motivation.

In short, this:

  • Migrates the library from Poetry to uv (via merging in python-dev)
  • Migrates from black to ruff format
  • Adds a uv workspace to the root folder
  • Changes the Makefile to invoke uv for every Python-related call
    • This simplifies the Makefile substantially as uv handles Python versions and package installations
  • Adds a pre-commit hook that invokes jing (RelaxNG schema validation) rather than relying on Makefile for this, as this means all core build checks can be invoked via pre-commit
  • Adds integration tests to check-build workflow (since data changes can also affect them)
  • Fixes a regression with Python 3.14 in create_extra_bib.py

Probably requires updating some documentation too which I haven't done yet.

@mbollmann mbollmann requested review from akoehn and mjpost February 1, 2026 19:12
@codecov
Copy link

codecov bot commented Feb 1, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.49%. Comparing base (f4a5401) to head (06eeee1).
⚠️ Report is 128 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7394      +/-   ##
==========================================
+ Coverage   95.46%   95.49%   +0.03%     
==========================================
  Files          35       35              
  Lines        3150     3171      +21     
==========================================
+ Hits         3007     3028      +21     
  Misses        143      143              
Files with missing lines Coverage Δ
python/acl_anthology/__init__.py 100.00% <100.00%> (ø)
python/acl_anthology/collections/collection.py 98.46% <100.00%> (ø)
python/acl_anthology/collections/event.py 99.23% <100.00%> (ø)
python/acl_anthology/collections/paper.py 92.71% <100.00%> (+0.04%) ⬆️
python/acl_anthology/collections/volume.py 96.61% <100.00%> (+0.01%) ⬆️
python/acl_anthology/exceptions.py 100.00% <100.00%> (ø)
python/acl_anthology/files.py 100.00% <100.00%> (ø)
python/acl_anthology/people/person.py 97.93% <ø> (ø)
python/acl_anthology/utils/git.py 76.27% <ø> (ø)
python/acl_anthology/utils/text.py 96.77% <100.00%> (ø)
... and 1 more
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions
Copy link

github-actions bot commented Feb 1, 2026

Build successful. Some useful links:

This preview will be removed when the branch is merged.

Copy link
Member

@akoehn akoehn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would wait with merging this until we have merged #7115 (bootstrap 5, guessing the number on mobile ...). There I also touched the makefile and hugo handling.

Generally looks good.

@mbollmann mbollmann mentioned this pull request Feb 7, 2026
@mbollmann
Copy link
Member Author

From #7115, so I don't forget:

I wonder if the version definitions + installation of these common dependencies can be factored out too (in the Github workflows), to avoid repetition (since there are currently three workflows that need them).

Github composite actions sound like they could achieve this: https://docs.github.com/en/actions/tutorials/create-actions/create-a-composite-action

@mbollmann
Copy link
Member Author

@mjpost Did 9f913d1 go through the library? The abstract contains unnecessary HTML entities ("), causing the integration tests here to fail.

I suppose this should be added to the main build checks, rather than causing surprises in the Python integration checks. Will need to think about this and might add a suggestion to this PR.

@mjpost
Copy link
Member

mjpost commented Feb 18, 2026

@mjpost Did 9f913d1 go through the library? The abstract contains unnecessary HTML entities ("), causing the integration tests here to fail.

No, it was a weird one-off where I had to add the paper manually. I thought it was best to just add it by hand but I can see I should have just handed this with a short library-based script. Noted for the future.

@mbollmann mbollmann mentioned this pull request Feb 18, 2026
17 tasks
@mbollmann
Copy link
Member Author

Assuming the build checks pass, this is ready for review, as I have added all important things here now, I think. @mjpost @akoehn

Sorry that it’s a bit messy due to changes from python-dev being merged in here, we could mitigate that with a separate merge python-dev->master first if you prefer.

Copy link
Member

@akoehn akoehn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not go into the python files. If there is something in there, let me know. Might have time for a proper review on the weekend.

All of these dependencies can be installed via Homebrew:

```bash
brew install bibutils hugo jing-trang just sass/sass/sass uv
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You remove the apt instructions and add homebrew -- don't think I don't see what you are doing!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to add back apt instructions, but it looked to me like uv can't be installed that way so I opted for the thing that works with all the dependencies, and on both Mac and Linux.

I assume that opinionated Linux users will know to use their preferred package managers instead. ;)

formatting guidelines defined by the [`black`](https://black.readthedocs.io/)
tool.
4. Python files need to follow the lint rules covered by the
4. Python files should have a maximum line length of 90 and follow the linting
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Why don't we follow the black/ruff format without exceptions?)

## How to use
## For end users

Install via `pip`:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like you want uv here as well, spread the gospel :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure actually. uv is mostly a project manager, and uv pip install --system exists but isn't really a recommended use of uv AFAIK. I just want to highlight that the package is on PyPI, I don’t think I want to go into the depths of recommending how to set up a project or venv for it :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, you mean for global installation. In that case, I would also not recommend pip but refer to pipx / uvx instead.

Copy link
Member Author

@mbollmann mbollmann Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But those are (IIRC) only useful for packages exposing CLI commands, which we currently don’t do :)

(We could, though!)

@mbollmann
Copy link
Member Author

I tweaked some build scripts and also chose to bump Hugo to v0.156.0, since that version breaks our build by removing a deprecated variable we used in the RSS template, and introduces more deprecations that I thought I’d take care of while I was at it.

If someone wants to review this more thoroughly, I’d recommend (1) waiting for #7570 and #7585, (2) merging python-dev into master, (3) merging master back into here. Then we have a little less noise in here in terms of files changed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Switch build system to uv

3 participants