Skip to content

Conversation

dcreager
Copy link
Member

As part of #20598, we added is_identical_to methods to TypeVarInstance and BoundTypeVarInstance, which compare when two typevar instances refer to "the same" underlying typevar, even if we have forced their lazy bounds/constraints as part of marking typevars as inferable. (Doing so results in a different salsa interned struct ID, since we've changed the contents of the bounds_or_constraints field.)

It turns out that marking typevars as inferable is not the only way that we might force lazy bounds/constraints; it also happens when we materialize a type containing a typevar. This surfaced as ecosystem report failures on #20677.

That means that we need a more long-term fix to this problem. (is_identical_to, and its underlying original field, were meant to be a temporary fix until we removed the MarkTypeVarsInferable type mapping.)

This PR extracts out a separate type (TypeVarIdentity) that only includes the fields that actually inform whether two typevars are "the same". All other properties of the typevar (default, bounds/constraints, etc) still live in TypeVarInstance. Call sites that care about typevar identity can now either store just TypeVarIdentity (if they never need access to those other properties), or continue to store TypeVarInstance but pull out its identity when performing those "are they the same typevar" comparisons. (All of this also applies respectively to BoundTypeVar{Identity,Instance}.) In particular, constraint sets now work on BoundTypeVarIdentity, and generic contexts still store a BoundTypeVarInstance (since we might need access to defaults when specializing), but are keyed on BoundTypeVarIdentity.

- Create TypeVarIdentity salsa-interned type with name, definition, and kind
- Create BoundTypeVarIdentity for identity comparisons
- Update TypeVarInstance to use identity field instead of separate fields
- Update GenericContext and SpecializationBuilder to use BoundTypeVarIdentity as map keys
- Remove is_identical_to methods in favor of direct identity comparison
- All tests pass (271 passed, 0 failed)

This ensures that typevars with the same logical identity but different
materialized bounds are recognized as identical, fixing the regression
where method calls on Top-materialized types fail.

Generated with Claude Code assistance.
Replace tuple value type in GenericContext.variables_inner with a proper
struct. This change:
- Renames GenericContextTypeVarOptions to GenericContextTypeVar
- Moves BoundTypeVarInstance into the struct alongside should_promote_literals
- Updates all methods that use variables_inner (promote_literals, merge,
  variables, heap_size, and SpecializationBuilder.build)

This makes the code cleaner and more maintainable by using a proper struct
instead of a tuple for the map values.
The original field was used to track the identity of typevars that were
transformed via mark_typevars_inferable, allowing them to be recognized
as the same typevar. This was needed for the old is_identical_to methods.

Since we now use TypeVarIdentity for identity comparisons and have removed
the is_identical_to methods, the original field is no longer needed. It was
only being passed through but never actually read or used.

This simplifies TypeVarInstance::new by removing an unused parameter.
Update DisplayBoundTypeVarInstance to DisplayBoundTypeVarIdentity and move
the display() method from BoundTypeVarInstance to BoundTypeVarIdentity.

Since the display implementation only uses the identity fields (name and
binding context), it makes more sense for it to operate on the identity
type directly. This also reduces coupling to the full BoundTypeVarInstance.

Updated all callers to use .identity(db).display(db) when displaying a
BoundTypeVarInstance.
Change ConstrainedTypeVar to store BoundTypeVarIdentity instead of
BoundTypeVarInstance. This makes constraints independent of typevar
materialization, which is appropriate since constraints should only care
about the logical identity of a typevar, not its specific bounds.

Updated:
- ConstrainedTypeVar::typevar field to BoundTypeVarIdentity
- ConstraintSet::range() and negated_range() to accept BoundTypeVarIdentity
- ConstrainedTypeVar::new_node() signature
- All callers in function.rs to pass .identity(db)
- Display code which now receives BoundTypeVarIdentity directly
@dcreager dcreager added internal An internal refactor or improvement ty Multi-file analysis & type inference labels Oct 11, 2025
Copy link
Contributor

github-actions bot commented Oct 11, 2025

Diagnostic diff on typing conformance tests

Changes were detected when running ty on typing conformance tests
--- old-output.txt	2025-10-13 23:49:17.110848042 +0000
+++ new-output.txt	2025-10-13 23:49:20.423865224 +0000
@@ -1,5 +1,5 @@
-fatal[panic] Panicked at /home/runner/.cargo/git/checkouts/salsa-e6f3bb7c2a062968/29ab321/src/function/execute.rs:217:25 when checking `/home/runner/work/ruff/ruff/typing/conformance/tests/aliases_type_statement.py`: `PEP695TypeAliasType < 'db >::value_type_(Id(cc17)): execute: too many cycle iterations`
-fatal[panic] Panicked at /home/runner/.cargo/git/checkouts/salsa-e6f3bb7c2a062968/29ab321/src/function/execute.rs:217:25 when checking `/home/runner/work/ruff/ruff/typing/conformance/tests/aliases_typealiastype.py`: `infer_definition_types(Id(1603f)): execute: too many cycle iterations`
+fatal[panic] Panicked at /home/runner/.cargo/git/checkouts/salsa-e6f3bb7c2a062968/29ab321/src/function/execute.rs:217:25 when checking `/home/runner/work/ruff/ruff/typing/conformance/tests/aliases_type_statement.py`: `PEP695TypeAliasType < 'db >::value_type_(Id(d017)): execute: too many cycle iterations`
+fatal[panic] Panicked at /home/runner/.cargo/git/checkouts/salsa-e6f3bb7c2a062968/29ab321/src/function/execute.rs:217:25 when checking `/home/runner/work/ruff/ruff/typing/conformance/tests/aliases_typealiastype.py`: `infer_definition_types(Id(1643f)): execute: too many cycle iterations`
 _directives_deprecated_library.py:15:31: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `int`
 _directives_deprecated_library.py:30:26: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `str`
 _directives_deprecated_library.py:36:41: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@__add__`

Copy link
Contributor

github-actions bot commented Oct 11, 2025

mypy_primer results

Changes were detected when running on open source projects
sympy (https://github.com/sympy/sympy)
- sympy/polys/polyclasses.py:1289:31: warning[unused-ignore-comment] Unused blanket `type: ignore` directive
- Found 11030 diagnostics
+ Found 11029 diagnostics
No memory usage changes detected ✅

* main:
  [ty] bidirectional type inference using function return type annotations (#20528)
  [ty] use type context more aggressively to infer values ​​when constructing a `TypedDict` (#20806)
Copy link
Contributor

github-actions bot commented Oct 11, 2025

ecosystem-analyzer results

Lint rule Added Removed Changed
unused-ignore-comment 0 1 0
Total 0 1 0

Full report with detailed diff (timing results)

Copy link
Contributor

@sharkdp sharkdp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, thank you!

It looks like we can now revert @carljm's change in signatures.rs and potentially reclaim some performance?

diff --git a/crates/ty_python_semantic/src/types/signatures.rs b/crates/ty_python_semantic/src/types/signatures.rs
index 039b89a6eb..108df8cb00 100644
--- a/crates/ty_python_semantic/src/types/signatures.rs
+++ b/crates/ty_python_semantic/src/types/signatures.rs
@@ -1292,19 +1292,8 @@ impl<'db> Parameters<'db> {
                     let class = nearest_enclosing_class(db, index, scope_id).unwrap();
 
                     Some(
-                        // It looks like unnecessary work here that we create the implicit Self
-                        // annotation using non-inferable typevars and then immediately apply
-                        // `MarkTypeVarsInferable` to it. However, this is currently necessary to
-                        // ensure that implicit-Self and explicit Self annotations are both treated
-                        // the same. Marking type vars inferable will cause reification of lazy
-                        // typevar defaults/bounds/constraints; this needs to happen for both
-                        // implicit and explicit Self so they remain the "same" typevar.
-                        typing_self(db, scope_id, typevar_binding_context, class, &Type::NonInferableTypeVar)
-                            .expect("We should always find the surrounding class for an implicit self: Self annotation").apply_type_mapping(
-                                db,
-                                &TypeMapping::MarkTypeVarsInferable(None),
-                                TypeContext::default()
-                            )
+                        typing_self(db, scope_id, typevar_binding_context, class, &Type::TypeVar)
+                            .expect("We should always find the surrounding class for an implicit self: Self annotation")
                     )
                 } else {
                     // For methods of non-generic classes that are not otherwise generic (e.g. return `Self` or

@AlexWaygood AlexWaygood removed their request for review October 13, 2025 08:07
@dcreager
Copy link
Member Author

dcreager commented Oct 13, 2025

It looks like we can now revert @carljm's change in signatures.rs and potentially reclaim some performance?

I currently have that being removed in #20677, where we remove the separate enum variants for inferable vs non-inferable. But I can try it here too and see if everything still passes.

* main: (25 commits)
  [ty] Diagnostic for generic classes that reference typevars in enclosing scope (#20822)
  Update Python compatibility from 3.13 to 3.14 in README.md (#20852)
  [syntax-errors]: break outside loop F701 (#20556)
  [ty] Treat `Callable`s as bound-method descriptors in special cases (#20802)
  [ty] Do not bind self to non-positional parameters (#20850)
  Fix syntax error false positives on parenthesized context managers (#20846)
  [ty] Remove 'pre-release software' warning (#20817)
  Render unsupported syntax errors in formatter tests (#20777)
  [ty] Treat functions, methods, and dynamic types as function-like `Callable`s (#20842)
  [ty] Move logic for `super()` inference to a new `types::bound_super` submodule (#20840)
  [ty] Fix false-positive diagnostics on `super()` calls (#20814)
  [ty] Move `class_member` to `member` module (#20837)
  [`ruff`] Use DiagnosticTag for more flake8 and numpy rules (#20758)
  [ty] Prefer declared base class attribute over inferred attribute on subclass (#20764)
  [ty] Log files that are slow to type check (#20836)
  Update cargo-bins/cargo-binstall action to v1.15.7 (#20827)
  Update CodSpeedHQ/action action to v4.1.1 (#20828)
  Update Rust crate pyproject-toml to v0.13.7 (#20835)
  Update Rust crate anstream to v0.6.21 (#20829)
  Update Rust crate libc to v0.2.177 (#20832)
  ...
@dcreager
Copy link
Member Author

But I can try it here too and see if everything still passes.

Removing this here added back a no-matching-overload ecosystem diagnostic that had been removed. I'm going to keep this removal on #20677, where I think it will be fully redundant.

@dcreager dcreager force-pushed the dcreager/typevar-identity branch from b934d4c to 0e53996 Compare October 13, 2025 23:47
@dcreager dcreager merged commit 5e08e54 into main Oct 14, 2025
41 checks passed
@dcreager dcreager deleted the dcreager/typevar-identity branch October 14, 2025 00:09
dcreager added a commit that referenced this pull request Oct 14, 2025
…rable

* origin/main: (26 commits)
  [ty] Add separate type for typevar "identity" (#20813)
  [ty] Diagnostic for generic classes that reference typevars in enclosing scope (#20822)
  Update Python compatibility from 3.13 to 3.14 in README.md (#20852)
  [syntax-errors]: break outside loop F701 (#20556)
  [ty] Treat `Callable`s as bound-method descriptors in special cases (#20802)
  [ty] Do not bind self to non-positional parameters (#20850)
  Fix syntax error false positives on parenthesized context managers (#20846)
  [ty] Remove 'pre-release software' warning (#20817)
  Render unsupported syntax errors in formatter tests (#20777)
  [ty] Treat functions, methods, and dynamic types as function-like `Callable`s (#20842)
  [ty] Move logic for `super()` inference to a new `types::bound_super` submodule (#20840)
  [ty] Fix false-positive diagnostics on `super()` calls (#20814)
  [ty] Move `class_member` to `member` module (#20837)
  [`ruff`] Use DiagnosticTag for more flake8 and numpy rules (#20758)
  [ty] Prefer declared base class attribute over inferred attribute on subclass (#20764)
  [ty] Log files that are slow to type check (#20836)
  Update cargo-bins/cargo-binstall action to v1.15.7 (#20827)
  Update CodSpeedHQ/action action to v4.1.1 (#20828)
  Update Rust crate pyproject-toml to v0.13.7 (#20835)
  Update Rust crate anstream to v0.6.21 (#20829)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ecosystem-analyzer internal An internal refactor or improvement ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants