chore: refactor pyproject

[SemyonSinchenko]

Reason for this PR

pytest should never be a part of the distribution, pyarrow should be an explicit dependency (see #836 ), etc.

Close #836

What changes are included in this PR?

Bump versions of actions, refactoring of the pyproject

Are these changes tested?

I build the wheel inside the Ubuntu 24.04 container, updated it with auditwheel in the same wake like in the CI-build. I was able to install it on Fedora f44 inside a fresh venv: all the dependencies including pyarrow were resolved correctly, test commands pass.

(.venv2) sem@fedora:~/github/GraphAr/python$ graphar show -p ../testing/ldbc/ldbc.graph.yml
INFO     edges:
           - tag_hasType_tagclass.edge.yaml
           - tagclass_isSubclassOf_tagclass.edge.yaml
           - forum_hasMember_person.edge.yaml
           - forum_containerOf_post.edge.yaml
           - forum_hasTag_tag.edge.yaml
           - comment_hasTag_tag.edge.yaml
           - place_isPartOf_place.edge.yaml
           - comment_replyOf_post.edge.yaml
           - organisation_isLocatedIn_place.edge.yaml
           - comment_replyOf_comment.edge.yaml
           - post_hasCreator_person.edge.yaml
           - comment_isLocatedIn_place.edge.yaml
           - person_hasInterest_tag.edge.yaml
           - person_workAt_organisation.edge.yaml
           - post_isLocatedIn_place.edge.yaml
           - post_hasTag_tag.edge.yaml
           - person_studyAt_organisation.edge.yaml
           - forum_hasModerator_person.edge.yaml
           - person_likes_post.edge.yaml
           - person_likes_comment.edge.yaml
           - comment_hasCreator_person.edge.yaml
           - person_isLocatedIn_place.edge.yaml
           - person_knows_person.edge.yaml
         name: ldbc
         prefix: /var/home/sem/github/GraphAr/testing/ldbc/
         version: gar/v1
         vertices:
           - tagclass.vertex.yaml
           - post.vertex.yaml
           - comment.vertex.yaml
           - place.vertex.yaml
           - tag.vertex.yaml
           - organisation.vertex.yaml
           - forum.vertex.yaml
           - person.vertex.yaml

(.venv2) sem@fedora:~/github/GraphAr/python$

Are there any user-facing changes?

No. Python was not published yet.

Checklist

• I have performed a self-review of my own code.
• I have formatted my own code using make cpplint before submitting when changed files are in the cpp directory.
• [x I have performed pre-commit run before commit the changed files.
• I have added tests to prove my changes are effective.

[SemyonSinchenko]

pre-commit is fixed in #935

[keksmd]

Thanks for taking this on — linking against the bundled pyarrow (#836) is a real pain point, and the Linux rpath change with the note about DT_RUNPATH resolution not being transitive is exactly right.

Both CI jobs are currently red, and they look like two distinct real failures rather than infrastructure noise. Details below, plus a few things I think the PR changes without meaning to.

Blockers

1. ARROW_VERSION: "0.24.1" is not an Arrow apt version

E: Version '0.24.1' for 'libarrow-dev' was not found

The Apache Arrow apt repository publishes versions shaped like 24.0.1-1, not 0.24.1, so the Ubuntu job fails after ~30s in "Install dependencies".

The # single source of version for arrow comment also isn't quite accurate yet: pyproject.toml carries its own independent pyarrow ~= 24.0 constraint, and nothing ties the two numbers together. If the goal is a single source, it would be worth deriving the apt pin from the pyarrow version (or vice versa) instead of restating it by hand in a different format.

2. The new graphar import step on macOS cannot pass with the Homebrew Arrow

ERROR    Import failed: Unsupported file type: orc

testing/neo4j/data/import.mini.yml references Movie.orc. ORC support is compiled behind #ifdef ARROW_ORC (python/src/bindings/utils/import_util.h), and ARROW_ORC is disabled in the Homebrew apache-arrow formula — which this repo already documents in cpp/README.md:

Currently, the Arrow C++ library has disabled ARROW_ORC in the brew formula, so you need to build and install the Arrow C++ library manually (with -DARROW_ORC=True).

So the step was added to a job where it can't go green as configured. Options: build Arrow with -DARROW_ORC=True on macOS, drop import from the macOS job, or use a mini fixture without an ORC input.

3. libarrow-acero-dev and libparquet-dev were dropped from the install without explanation

libarrow-dataset-dev pulls acero in transitively, but Parquet is a separate package and import_util.h calls GetDataFromParquetFile. The Ubuntu job dies before linking, so this is untested either way — it will only be observable once (1) is fixed.

Unintended regressions

4. The package extras are removed, not just the pytest runtime dependency

The build backend here is scikit_build_core.build, not Poetry. That means [tool.poetry.group.test] and [tool.poetry.group.docs] are only usable by the Poetry CLI inside this repo — they do not appear in the built wheel. Deleting [project.optional-dependencies] therefore removes both graphar[test] and graphar[docs] from what gets published.

Getting pytest out of the runtime dependencies is clearly correct. Dropping the extras is a separate, user-visible change; if it is intended, it deserves a line in the description.

5. Documentation goes stale in the same commit

python/README.md still instructs:

pip install -e "./python[test]"

That extra no longer exists after this PR.

6. requires-python goes >=3.9 to >=3.10

That is a breaking change for downstream consumers, and the PR is labelled chore: with no mention of it. Worth calling out in the description and in release notes.

Smaller things

7. Dead statement in python/CMakeLists.txt

Inside if(_graphar_macos_rpaths):

set_target_properties(graphar PROPERTIES INSTALL_RPATH "${_graphar_macos_rpaths_joined}")

is immediately overwritten by the unconditional

set_target_properties(graphar PROPERTIES INSTALL_RPATH "${_graphar_rpath}")

a few lines later. The first call has no effect and can be removed.

8. Build-machine Arrow paths get baked into the shipped artifact

_graphar_macos_rpaths_joined ends up in the installed target's INSTALL_RPATH, while the comment right above it states those directories "are absent on the user's machine". That embeds the builder's absolute paths into the wheel and hurts reproducibility. Since BUILD_WITH_INSTALL_RPATH ON is what makes linking work here, it may be worth separating the link-time search path from the installed rpath.

9. pytest test -v to pytest -vvv loses the explicit test directory

Collection now starts from rootdir instead of test/. That widens what gets collected while only increasing verbosity.

10. The job is renamed to "Ubuntu 24.04" but runs-on stays ubuntu-latest

The name will silently stop being true the next time ubuntu-latest moves. Either pin runs-on: ubuntu-24.04 or keep the generic name.

11. Checklist nits

"I have added tests to prove my changes are effective" is unchecked, and two checklist entries are written as [x without the closing bracket, so they don't render as checked.

[SemyonSinchenko]

Hi @SYaoJun @yangxk1 ! This works on all the tests I did locally and it should resolve the problem of conflict between bundled with GAR arrow and installed pyarrow. I also fixed some small warnings and changed some parts of the pyproject to follow best practices.

Please, take a look when you have time. Few ppl already asked me about the status of it and are awaiting it. Thanks in advance!

cc: @adsharma

[SYaoJun]

LGTM

[adsharma]

Thank you for taking this on and fixing it @SemyonSinchenko!