Skip to content

[ty] Improve support for Callable type context#23888

Open
ibraheemdev wants to merge 1 commit intomainfrom
ibraheem/callable-tcx
Open

[ty] Improve support for Callable type context#23888
ibraheemdev wants to merge 1 commit intomainfrom
ibraheem/callable-tcx

Conversation

@ibraheemdev
Copy link
Member

Improves literal promotion and generic call inference that involve Callable type context. Resolves astral-sh/ty#3016.

@ibraheemdev ibraheemdev added the ty Multi-file analysis & type inference label Mar 11, 2026
return_ty: self
.return_ty
.apply_type_mapping_impl(db, type_mapping, tcx, visitor),
if let TypeMapping::UniqueSpecialization { specialization } = type_mapping {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should all be removed by #23848, but it also gives us some useful test cases in the meantime. cc @dcreager.

@astral-sh-bot
Copy link

astral-sh-bot bot commented Mar 11, 2026

Typing conformance results

No changes detected ✅

Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 85.05%. The percentage of expected errors that received a diagnostic held steady at 78.05%. The number of fully passing files held steady at 63/132.

@astral-sh-bot
Copy link

astral-sh-bot bot commented Mar 11, 2026

mypy_primer results

Changes were detected when running on open source projects
attrs (https://github.com/python-attrs/attrs)
- tests/test_validators.py:787:21: error[invalid-argument-type] Argument to function `and_` is incorrect: Expected `(Any, Attribute[int], int, /) -> Any`, found `(Any, Attribute[Literal[10]], Literal[10], /) -> Any`
- Found 671 diagnostics
+ Found 670 diagnostics

@astral-sh-bot
Copy link

astral-sh-bot bot commented Mar 11, 2026

Memory usage report

Summary

Project Old New Diff Outcome
flake8 47.90MB 47.89MB -0.03% (16.64kB) ⬇️
trio 117.80MB 117.76MB -0.03% (32.15kB) ⬇️
sphinx 265.18MB 265.05MB -0.05% (136.52kB) ⬇️
prefect 702.31MB 702.11MB -0.03% (197.68kB) ⬇️

Significant changes

Click to expand detailed breakdown

flake8

Name Old New Diff Outcome
inferable_typevars_inner 26.00kB 23.78kB -8.56% (2.23kB) ⬇️
infer_scope_types_impl 1004.28kB 1002.12kB -0.21% (2.16kB) ⬇️
infer_expression_types_impl 1.07MB 1.07MB -0.14% (1.58kB) ⬇️
Type<'db>::try_call_dunder_get_ 373.83kB 372.77kB -0.28% (1.06kB) ⬇️
StaticClassLiteral<'db>::try_mro_ 335.69kB 334.64kB -0.31% (1.05kB) ⬇️
FunctionType 433.78kB 432.78kB -0.23% (1.00kB) ⬇️
Type<'db>::class_member_with_policy_ 547.17kB 546.45kB -0.13% (740.00B) ⬇️
infer_definition_types 1.86MB 1.86MB -0.03% (660.00B) ⬇️
CallableType 166.08kB 165.52kB -0.34% (576.00B) ⬇️
Type<'db>::apply_specialization_ 216.62kB 216.09kB -0.25% (548.00B) ⬇️
FunctionType<'db>::signature_ 358.91kB 358.38kB -0.15% (544.00B) ⬇️
Type<'db>::class_member_with_policy_::interned_arguments 300.32kB 299.81kB -0.17% (520.00B) ⬇️
is_equivalent_to_object_inner 26.32kB 25.92kB -1.51% (408.00B) ⬇️
BoundMethodType<'db>::into_callable_type_ 26.96kB 26.60kB -1.35% (372.00B) ⬇️
StaticClassLiteral<'db>::try_mro_::interned_arguments 76.08kB 75.73kB -0.46% (360.00B) ⬇️
... 22 more

trio

Name Old New Diff Outcome
infer_expression_types_impl 7.06MB 7.05MB -0.08% (5.74kB) ⬇️
inferable_typevars_inner 68.55kB 64.05kB -6.56% (4.50kB) ⬇️
infer_expression_type_impl 1.43MB 1.42MB -0.23% (3.40kB) ⬇️
FunctionType 1.50MB 1.50MB -0.14% (2.16kB) ⬇️
infer_definition_types 7.57MB 7.57MB -0.03% (2.13kB) ⬇️
StaticClassLiteral<'db>::try_mro_ 863.60kB 861.81kB -0.21% (1.79kB) ⬇️
Type<'db>::try_call_dunder_get_ 1.37MB 1.37MB -0.12% (1.68kB) ⬇️
Type<'db>::class_member_with_policy_ 1.98MB 1.98MB -0.07% (1.45kB) ⬇️
infer_scope_types_impl 4.79MB 4.79MB -0.03% (1.34kB) ⬇️
FunctionType<'db>::signature_ 1.07MB 1.07MB -0.11% (1.17kB) ⬇️
Specialization 474.62kB 473.53kB -0.23% (1.09kB) ⬇️
infer_unpack_types 147.74kB 146.71kB -0.70% (1.03kB) ⬇️
Type<'db>::apply_specialization_ 737.55kB 736.57kB -0.13% (1004.00B) ⬇️
Type<'db>::class_member_with_policy_::interned_arguments 1.10MB 1.10MB -0.08% (936.00B) ⬇️
code_generator_of_static_class 210.72kB 209.87kB -0.41% (876.00B) ⬇️
... 31 more

sphinx

Name Old New Diff Outcome
infer_expression_types_impl 21.50MB 21.47MB -0.13% (27.53kB) ⬇️
infer_definition_types 24.00MB 23.98MB -0.08% (18.60kB) ⬇️
infer_scope_types_impl 15.59MB 15.58MB -0.07% (10.46kB) ⬇️
FunctionType 3.12MB 3.11MB -0.27% (8.77kB) ⬇️
Type<'db>::class_member_with_policy_ 7.55MB 7.55MB -0.10% (7.78kB) ⬇️
StaticClassLiteral<'db>::try_mro_ 2.11MB 2.10MB -0.33% (7.19kB) ⬇️
Type<'db>::try_call_dunder_get_ 4.94MB 4.93MB -0.12% (6.09kB) ⬇️
inferable_typevars_inner 81.07kB 75.31kB -7.10% (5.75kB) ⬇️
CallableType 1.08MB 1.07MB -0.52% (5.70kB) ⬇️
Type<'db>::class_member_with_policy_::interned_arguments 3.99MB 3.98MB -0.12% (4.77kB) ⬇️
FunctionType<'db>::signature_ 2.28MB 2.27MB -0.20% (4.76kB) ⬇️
infer_unpack_types 445.72kB 442.06kB -0.82% (3.66kB) ⬇️
Type<'db>::apply_specialization_ 1.66MB 1.66MB -0.19% (3.29kB) ⬇️
code_generator_of_static_class 529.26kB 526.57kB -0.51% (2.69kB) ⬇️
cached_protocol_interface 196.69kB 194.00kB -1.36% (2.68kB) ⬇️
... 31 more

prefect

Name Old New Diff Outcome
infer_definition_types 88.50MB 88.45MB -0.05% (48.62kB) ⬇️
infer_expression_types_impl 60.46MB 60.41MB -0.07% (44.16kB) ⬇️
infer_scope_types_impl 52.70MB 52.68MB -0.04% (21.23kB) ⬇️
Type<'db>::class_member_with_policy_ 17.26MB 17.25MB -0.05% (8.33kB) ⬇️
inferable_typevars_inner 161.37kB 153.72kB -4.74% (7.65kB) ⬇️
infer_expression_type_impl 14.28MB 14.27MB -0.05% (7.64kB) ⬇️
FunctionType 8.45MB 8.45MB -0.06% (5.47kB) ⬇️
infer_unpack_types 877.57kB 872.15kB -0.62% (5.41kB) ⬇️
Type<'db>::try_call_dunder_get_ 10.45MB 10.44MB -0.05% (5.39kB) ⬇️
CallableType 1.89MB 1.89MB -0.24% (4.57kB) ⬇️
Type<'db>::class_member_with_policy_::interned_arguments 9.32MB 9.32MB -0.05% (4.47kB) ⬇️
StaticClassLiteral<'db>::try_mro_ 6.03MB 6.02MB -0.06% (3.96kB) ⬇️
FunctionType<'db>::signature_ 3.86MB 3.86MB -0.08% (3.20kB) ⬇️
code_generator_of_static_class 1.80MB 1.79MB -0.16% (2.96kB) ⬇️
StaticClassLiteral<'db>::implicit_attribute_inner_ 9.72MB 9.71MB -0.03% (2.96kB) ⬇️
... 30 more

@astral-sh-bot
Copy link

astral-sh-bot bot commented Mar 11, 2026

ecosystem-analyzer results

Lint rule Added Removed Changed
invalid-argument-type 0 1 0
Total 0 1 0

Full report with detailed diff (timing results)

@ibraheemdev ibraheemdev force-pushed the ibraheem/callable-tcx branch from b229936 to 7c2c8ee Compare March 11, 2026 13:07
@codspeed-hq
Copy link

codspeed-hq bot commented Mar 11, 2026

Merging this PR will improve performance by 4.95%

⚡ 1 improved benchmark
✅ 25 untouched benchmarks
⏩ 30 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
WallTime static_frame 30.4 s 29 s +4.95%

Comparing ibraheem/callable-tcx (7c2c8ee) with main (22f63ec)

Open in CodSpeed

Footnotes

  1. 30 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Copy link
Contributor

@carljm carljm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few comments. @dcreager I'm wondering if you'd be a better reviewer here, as you'd be better equipped to catch if this is doing anything that will cause trouble down the line for the new constraint solver.

/// This is the case for any type which may contain types in non-covariant position within it,
/// e.g., nominal instances of a generic class, or callables.
pub(crate) fn may_prefer_declared_type(self, db: &'db dyn Db) -> bool {
self.class_specialization(db).is_some() || self.is_callable_type()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we'd better resolve type aliases here? Otherwise a type alias to a callable type as type context won't be recognized. (Let's add a test).

Comment on lines +909 to +910
// Note that unique specializations are typically applied to type context, so we only
// care about the standard `Callable` form here.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A normal Callable annotation can result in a ParametersKind::Gradual (Callable[..., int]) or a ParametersKind::ParamSpec (Callable[P, R] where P is a ParamSpec). So I'm not sure this rationale makes sense as written?

I think ignoring Gradual still makes sense, since it can't contribute any useful type context, but ignoring ParamSpec I think means we won't handle ParamSpec-using wrappers correctly, e.g.

def wrap[**P, T](f: Callable[P, T]) -> Callable[P, T]: ...
y: Callable[[Any], int] = wrap(takes_int) # still reveals `(x: int) -> int`


f(parameter.annotated_type(), variance);

visitor.visit(self, || {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't make repeated calls using visitor.visit(self, ...) like this, because CycleDetector caches repeat visits, based on the first-argument key. So only the first visit here will actually run. This causes us to lose later non-covariant occurrences, e.g. def make[T](x: T) -> Callable[[int], list[T]] reveals (int, /) -> list[Literal[1]].

@carljm carljm assigned dcreager and unassigned carljm Mar 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ecosystem-analyzer ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Promote literals in contravariant position, not just invariant

3 participants