Move TypeHint into a Python expression AST by Tpt · Pull Request #5671 · PyO3/pyo3

Tpt · 2025-12-02T09:46:13Z

Implement a subset of Python expr AST

Adds to the "name" construct a kind to distinguish global (global name that does not need to be resolved against the current module) vs local (names that needs to be resolved).
This consolidates the previous builtin/local/module construct by taking into account that modules can be relative (for example if in a #[pyclass] the user set module= to the submodule name)

Adds also the support for Callable[[int], float] and a beginning of constructs to represent constants (currently only None but will be useful for typing.Literal support)

The interesting part of this PR is the inspect/mod.rs file

Implement a subset of Python expr AST Adds to the "name" construct a kind to distinguish global (global name that does not need to be resolved against the current module) vs local (names that needs to be resolved). This consolidates the previous builtin/local/module construct by taking into account that modules can be relative (for example if in a #[pyclass] the user set `module=` to the submodule name) Adds also the support for `Callable[[int], float]` and a beginning of constructs to represent constants (currently only None but will be useful for typing.Literal support) TODO: - update the macro code - update the introspection code

davidhewitt

Thanks for working on this.

My initial reaction reading the code is that PyStaticExpr is the right data format but the API for for type hints specifically is less nice, e.g. compare the union constructor we had before with the bit_or constructor we have now.

Due to the limitations of const fn (no allocations etc) I think the only way to get the ergonomics back would be a macro in that case, e.g. type_hint_union!(&a, &b, &c) instead of PyStaticExpr(&a, &PyStaticExpr::bit_or(&b, &c)).

Maybe for others like TypeHint::module_attr it might be able to have a normal const fn if we wanted to keep that API.

I'm also not sure if we should keep struct TypeHint(PyStaticExpr) as a way to have a namespace for type hint constructors specifically.

Wonder what you think of those kinds of points?

davidhewitt · 2025-12-05T08:31:21Z

src/inspect/mod.rs

+pub enum PyStaticConstant {
+    /// None
+    None,
+    // TODO: add Bool(bool), String(&'static str)... (is useful for Literal["foo", "bar"] types)


String(&'static str) would also be interesting because it can be used as an escape hatch for forward-reference style annotations for things we can't parse.

e.g. we could have "tuple[()]" already as a type hint until we can support it natively (with a PyStaticCall ast node, I guess).

yes for String(&'static str)! I am planning to do it in a follow up if it's fine with you (to get it done properly, it needs a JSON escaping implementation working in const...).

For tuple[()] it's possible with this MR with the ::Tuple(&[]) variant.

Tpt · 2025-12-05T14:13:55Z

Thank you!

Due to the limitations of const fn (no allocations etc) I think the only way to get the ergonomics back would be a macro in that case, e.g. type_hint_union!(&a, &b, &c) instead of PyStaticExpr(&a, &PyStaticExpr::bit_or(&b, &c)).

Yes! It might make sense to have them for usual operations (union and subscript). Will add them.

I'm also not sure if we should keep struct TypeHint(PyStaticExpr) as a way to have a namespace for type hint constructors specifically.

If we had to end up with macros anyway, then the "namespace for constructors" usecase seems less interesting to me. What I can do is experiment a bit with macros and see what it looks like?

davidhewitt · 2025-12-05T19:50:26Z

👍 sounds good to me, thanks!

Implement a subset of Python expr AST Adds to the "name" construct a kind to distinguish global (global name that does not need to be resolved against the current module) vs local (names that needs to be resolved). This consolidates the previous builtin/local/module construct by taking into account that modules can be relative (for example if in a #[pyclass] the user set `module=` to the submodule name) Adds also the support for `Callable[[int], float]` and a beginning of constructs to represent constants (currently only None but will be useful for typing.Literal support) TODO: - update the macro code - update the introspection code

Tpt · 2025-12-09T15:37:23Z

@davidhewitt Experiment done with macros. I have replaced the const constructor with them. Does it looks good to you? If yes, I will move toward the next steps (same kind of changes in macro and introspection code)

davidhewitt

The macro constructors seem to work nicely to me 👍

pyo3-introspection/src/stubs.rs

davidhewitt · 2025-12-12T11:11:16Z

src/impl_/wrap.rs

This seems justifiable to extract into a separate PR.

Tpt · 2025-12-12T13:09:24Z

The macro constructors seem to work nicely to me 👍

Amazing! Will finish this MR then.

Tpt · 2025-12-12T15:01:10Z

I don't think we are fully there yet in term of imports generation in the stubs: if the module argument of #[pyclass] is not absolute we might still generate wrong imports. A better approach might be to include in the PyTypeInfo::TYPE_HINT of classes generated by #[pyclass] the introspection identifier: this way the stub generator can build proper import paths for all of them with full knowledge of the module layout. We can assume all other type hints from traits to be absolute paths. I plan to do it in a follow up.

Then for custom type hints, I guess the best approach is to ask the user to write imports from them in the #[pymodule] macro. This way the imports generation algorithm is:

For type hints from #[pyclass] generate proper module path from the library layout and edit the type hint
Start from import from #[pymodule]
For all type hints that are not from the user, generate an import for them if they do not conflict with existing imports and rewrite the type hint to leverage these imports.

This way we should hopefully get something correct. And PyStaticNameKind::Global means "in the python intrepreter root (either a module or a built-in)" and PyStaticNameKind::Local means "relative to importswritten #[pymodule]". The third case "with module path written by the user but maybe relative or wrong" does not exist anymore.

davidhewitt

Overall looks great, thanks! Small thought about the macro syntax, might not be a good idea or possible anyway, once we decide on that I think this is good to merge!

(Sorry for the delayed review, holidays & sickness...)

davidhewitt · 2026-01-09T13:56:31Z

src/inspect/mod.rs

+/// ```
+#[macro_export]
+macro_rules! type_hint_subscript {
+    ($l:expr, $r:expr) => {


I wonder if the syntax should have the []?

Suggested change

($l:expr, $r:expr) => {

($l:expr [ $r:expr ]) => {

Sadly with this syntax I get error: $l:expris followed by[, which is not allowed for expr fragments

davidhewitt · 2026-01-09T13:57:55Z

src/inspect/mod.rs

+#[macro_export]
+macro_rules! type_hint_union {
+    ($e:expr) => { $e };
+    ($l:expr , $($r:expr),+) => { $crate::inspect::PyStaticExpr::BinOp {


Similar to the wondering about the subscript macro, should this one use pipes? I feel about this one less strongly, Union[A, B] uses commas after all.

Suggested change

($l:expr , $($r:expr),+) => { $crate::inspect::PyStaticExpr::BinOp {

($l:expr | $($r:expr)|+) => { $crate::inspect::PyStaticExpr::BinOp {

Similarly I get ``$l:expris followed by|`, which is not allowed for `expr` fragments`

Tpt · 2026-01-09T15:14:48Z

@davidhewitt Thank you for the review! Your slow review gave me yet another reason to have a quiet Christmas break and I won't complain about it :)

Tpt added 2 commits December 1, 2025 16:27

WIP

c2b3270

davidhewitt reviewed Dec 5, 2025

View reviewed changes

Tpt added 2 commits December 9, 2025 16:34

Return type: remove one stage of the specialization (PyO3#5673)

ebc8f12

Introduces macros

8ab1862

Tpt force-pushed the tpt/hint-ast branch from 77efd1a to 8ab1862 Compare December 10, 2025 12:56

Tpt added 3 commits December 10, 2025 16:18

Align macros-backend API

22d90e7

Cleaner representation for setters

1a4580d

Implement the introspection side

a3d7c06

davidhewitt reviewed Dec 12, 2025

View reviewed changes

Merge remote-tracking branch 'upstream/HEAD' into tpt/hint-ast

2301fc5

Fixes tests

dac34fb

Tpt force-pushed the tpt/hint-ast branch from 6dcc82f to dac34fb Compare December 12, 2025 15:03

Tpt added 2 commits December 12, 2025 16:22

Merge remote-tracking branch 'upstream/HEAD' into tpt/hint-ast

93286e9

Nit

da6a950

Tpt marked this pull request as ready for review December 12, 2025 15:30

Tpt changed the title ~~Draft: Move TypeHint into a Python expression AST~~ Move TypeHint into a Python expression AST Dec 12, 2025

davidhewitt reviewed Jan 9, 2026

View reviewed changes

davidhewitt approved these changes Jan 10, 2026

View reviewed changes

davidhewitt added this pull request to the merge queue Jan 10, 2026

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 10, 2026

Tpt added this pull request to the merge queue Jan 12, 2026

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 12, 2026

Tpt added 2 commits January 12, 2026 11:09

Merge branch 'main' into tpt/hint-ast

0c3a38e

Fixes deleter type annotation

8dadfba

Tpt added this pull request to the merge queue Jan 12, 2026

Merged via the queue into PyO3:main with commit 7fb7dde Jan 12, 2026
41 of 43 checks passed

Tpt deleted the tpt/hint-ast branch January 12, 2026 12:05

	($l:expr , $($r:expr),+) => { $crate::inspect::PyStaticExpr::BinOp {
	($l:expr \| $($r:expr)\|+) => { $crate::inspect::PyStaticExpr::BinOp {

Conversation

Tpt commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

davidhewitt left a comment

Choose a reason for hiding this comment

Uh oh!

davidhewitt Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Tpt Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Tpt commented Dec 5, 2025

Uh oh!

davidhewitt commented Dec 5, 2025

Uh oh!

Tpt commented Dec 9, 2025

Uh oh!

davidhewitt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

davidhewitt Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Tpt commented Dec 12, 2025

Uh oh!

Tpt commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

davidhewitt left a comment

Choose a reason for hiding this comment

Uh oh!

davidhewitt Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Tpt Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

davidhewitt Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Tpt Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Tpt commented Jan 9, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Tpt commented Dec 2, 2025 •

edited

Loading

Tpt commented Dec 12, 2025 •

edited

Loading