docs: add guide chapter detailing current way to share classes between modules#5938
docs: add guide chapter detailing current way to share classes between modules#5938davidhewitt wants to merge 6 commits intoPyO3:mainfrom
Conversation
There was a problem hiding this comment.
Thank you for the great writing.
I tend to feel using the cpython API is sometime a bit simpler than relying on the C-API.
For example if base-package expose a constant __version__ = (major, minor, patch, abi) and then classes like:
class Foo:
@staticmethod
def __from_capsule__(capsule: CapsuleType) -> Self: ...
def __capsule__(self) -> CapsuleType: ...then external crates can just do py.import("base_package").getattr("__version__") and py.import("base_package").getattr("Foo").call_method1("__from_capsule__", (my_capsule,)) (the actual repr(C) type for Foo being defined in a shared base-package-ffi crate or something similar). This is the approach used by Arrow and some other libraries.
To me, it feels slightly simple and less error prone than the C-ABI way but does not change much.
| /// | ||
| /// In particular, note that Rust's default `#[repr(Rust)]` and `extern "Rust"` functions have no stability | ||
| /// guarantees, so storing and dereferencing a pointer to a Rust function in a capsule which was produced | ||
| /// by a separate compiled extension is likely UB. |
There was a problem hiding this comment.
I would drop the "likely", if I am not mistaken this is definitely UB in the general case.
| /// by a separate compiled extension is likely UB. | |
| /// by a separate compiled extension is UB. |
|
|
||
| At a minimum, this will probably include: | ||
|
|
||
| - `get_shared_type: extern "C" fn() -> Py<PyType>` - a function to get the `#[pyclass]` Python type object for `SharedType`. |
There was a problem hiding this comment.
Do we guarantee that Py is ffi-safe? If yes, it might be worth it to mention it in its documentation.
I had some time recently to investigate #1444 and plan how this might be one day addressed inside PyO3.
I came up with a protoype which shows a current (working) solution to this, with a bunch of limitations and caveats.
This PR writes up documentation about that prototype and proposes future work we could pursue to make this pattern easier in future. (It's hard, not supported well by
rustc,cargo, or us, so lots of surface for possible improvement.)