Skip to content

docs: add guide chapter detailing current way to share classes between modules#5938

Open
davidhewitt wants to merge 6 commits intoPyO3:mainfrom
davidhewitt:pyclass-sharing-doc
Open

docs: add guide chapter detailing current way to share classes between modules#5938
davidhewitt wants to merge 6 commits intoPyO3:mainfrom
davidhewitt:pyclass-sharing-doc

Conversation

@davidhewitt
Copy link
Copy Markdown
Member

@davidhewitt davidhewitt commented Apr 3, 2026

I had some time recently to investigate #1444 and plan how this might be one day addressed inside PyO3.

I came up with a protoype which shows a current (working) solution to this, with a bunch of limitations and caveats.

This PR writes up documentation about that prototype and proposes future work we could pursue to make this pattern easier in future. (It's hard, not supported well by rustc, cargo, or us, so lots of surface for possible improvement.)

@davidhewitt
Copy link
Copy Markdown
Member Author

Copy link
Copy Markdown
Contributor

@Tpt Tpt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the great writing.

I tend to feel using the cpython API is sometime a bit simpler than relying on the C-API.

For example if base-package expose a constant __version__ = (major, minor, patch, abi) and then classes like:

class Foo:
    @staticmethod
    def __from_capsule__(capsule: CapsuleType) -> Self: ...
    def __capsule__(self) -> CapsuleType: ...

then external crates can just do py.import("base_package").getattr("__version__") and py.import("base_package").getattr("Foo").call_method1("__from_capsule__", (my_capsule,)) (the actual repr(C) type for Foo being defined in a shared base-package-ffi crate or something similar). This is the approach used by Arrow and some other libraries.

To me, it feels slightly simple and less error prone than the C-ABI way but does not change much.

///
/// In particular, note that Rust's default `#[repr(Rust)]` and `extern "Rust"` functions have no stability
/// guarantees, so storing and dereferencing a pointer to a Rust function in a capsule which was produced
/// by a separate compiled extension is likely UB.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would drop the "likely", if I am not mistaken this is definitely UB in the general case.

Suggested change
/// by a separate compiled extension is likely UB.
/// by a separate compiled extension is UB.


At a minimum, this will probably include:

- `get_shared_type: extern "C" fn() -> Py<PyType>` - a function to get the `#[pyclass]` Python type object for `SharedType`.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we guarantee that Py is ffi-safe? If yes, it might be worth it to mention it in its documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants