Skip to content

Conversation

Gankra
Copy link
Contributor

@Gankra Gankra commented Oct 14, 2025

This is a second take at the implicit imports approach, allowing from . import submodule in an __init__.pyi to create the mypackage.submodule attribute everyhere.

This implementation operates inside of the available_submodule_attributes subsystem instead of as a re-export rule.

The upside of this is we are no longer purely syntactic, and absolute from imports that happen to target submodules work (an intentional discussed deviation from pyright which demands a relative from import). Also we don't re-export functions or classes.

The downside(?) of this is star imports no longer see these attributes (this may be either good or bad. I believe it's not a huge lift to make it work with star imports but it's some non-trivial reworking).

I've also intentionally made import mypackage.submodule not trigger this rule although it's trivial to change that.

I've tried to cover as many relevant cases as possible for discussion in the new test file I've added (there are some random overlaps with existing tests but trying to add them piecemeal felt confusing and weird, so I just made a dedicated file for this extension to the rules).

Fixes #133

Summary

Test Plan

This is a second take at the implicit imports approach, allowing `from . import submodule` in an `__init__.pyi` to create the `mypackage.submodule` attribute everyhere.

This implementation operates inside of the available_submodule_attributes subsystem instead of as a re-export rule.

The upside of this is we are no longer purely syntactic, and absolute from imports that happen to target submodules work (an intentional discussed deviation from pyright which demands a relative from import). Also we don't re-export functions or classes.

The downside(?) of this is star imports no longer see these attributes (this may be either good or bad. I believe it's not a huge lift to make it work with star imports but it's some non-trivial reworking).

I've also intentionally made `import mypackage.submodule` not trigger this rule although it's trivial to change that.

I've tried to cover as many relevant cases as possible for discussion in the new test file I've added (there are some random overlaps with existing tests but trying to add them piecemeal felt confusing and weird, so I just made a dedicated file for this extension to the rules).

Fixes #133
Copy link
Contributor

Diagnostic diff on typing conformance tests

No changes detected when running ty on typing conformance tests ✅

Copy link
Contributor

mypy_primer results

Changes were detected when running on open source projects
asynq (https://github.com/quora/asynq)
- asynq/async_task.py:78:24: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `scheduler`
- asynq/async_task.py:115:9: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `scheduler`
- asynq/contexts.py:59:19: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `scheduler`
+ asynq/contexts.py:59:19: error[unresolved-attribute] Type `<module 'asynq.scheduler'>` has no attribute `_state`
+ asynq/tests/test_debug.py:30:32: error[invalid-argument-type] Argument to function `dump_error` is incorrect: Expected `BaseException`, found `None`
+ asynq/tests/test_debug.py:40:46: error[invalid-argument-type] Argument to function `format_error` is incorrect: Expected `BaseException`, found `None`
- asynq/tests/test_debug.py:30:9: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
- asynq/tests/test_debug.py:40:21: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
- asynq/tests/test_debug.py:43:5: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
- asynq/tests/test_debug.py:46:25: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
- asynq/tests/test_debug.py:49:17: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
- asynq/tests/test_debug.py:57:17: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
- asynq/tests/test_debug.py:62:5: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
- asynq/tests/test_debug.py:65:17: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
- asynq/tests/test_debug.py:80:5: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
- asynq/tests/test_debug.py:81:17: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
- asynq/tests/test_debug.py:87:5: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
- asynq/tests/test_debug.py:88:17: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
- asynq/tests/test_debug.py:100:13: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
- asynq/tests/test_debug.py:115:32: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
+ asynq/tests/test_debug.py:199:60: error[invalid-argument-type] Argument to function `extract_tb` is incorrect: Expected `TracebackType`, found `None | @Todo`
- asynq/tests/test_debug.py:139:9: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
- asynq/tests/test_debug.py:199:37: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
- asynq/tests/test_debug.py:249:31: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
+ asynq/tests/test_debug.py:249:53: error[invalid-argument-type] Argument to function `format_tb` is incorrect: Expected `TracebackType`, found `None | @Todo`
- asynq/tests/test_debug.py:259:26: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
+ asynq/tests/test_debug.py:306:38: error[invalid-argument-type] Argument to function `filter_traceback` is incorrect: Expected `list[str]`, found `list[LiteralString]`
+ asynq/tests/test_debug.py:323:38: error[invalid-argument-type] Argument to function `filter_traceback` is incorrect: Expected `list[str]`, found `list[LiteralString]`
+ asynq/tests/test_debug.py:355:38: error[invalid-argument-type] Argument to function `filter_traceback` is incorrect: Expected `list[str]`, found `list[LiteralString]`
+ asynq/tests/test_debug.py:383:38: error[invalid-argument-type] Argument to function `filter_traceback` is incorrect: Expected `list[str]`, found `list[LiteralString]`
- asynq/tests/test_debug.py:306:9: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
- asynq/tests/test_debug.py:309:19: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
- asynq/tests/test_debug.py:323:9: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
- asynq/tests/test_debug.py:355:9: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
- asynq/tests/test_debug.py:383:9: error[unresolved-attribute] Type `<module 'asynq'>` has no attribute `debug`
- Found 139 diagnostics
+ Found 122 diagnostics

paasta (https://github.com/yelp/paasta)
+ paasta_tools/cli/cmds/validate.py:890:12: error[call-non-callable] Object of type `<module 'croniter.croniter'>` is not callable
- Found 913 diagnostics
+ Found 914 diagnostics

tornado (https://github.com/tornadoweb/tornado)
- tornado/test/circlerefs_test.py:146:24: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `testing`
- tornado/test/circlerefs_test.py:147:18: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `httpserver`
- tornado/test/circlerefs_test.py:205:18: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `concurrent`
- tornado/test/escape_test.py:217:22: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `escape`
+ tornado/test/escape_test.py:217:51: error[invalid-argument-type] Argument to function `linkify` is incorrect: Expected `bool`, found `Unknown | list[Unknown | str] | bool | str | ((href) -> Unknown)`
+ tornado/test/escape_test.py:217:51: error[invalid-argument-type] Argument to function `linkify` is incorrect: Expected `str | ((str, /) -> str)`, found `Unknown | list[Unknown | str] | bool | str | ((href) -> Unknown)`
+ tornado/test/escape_test.py:217:51: error[invalid-argument-type] Argument to function `linkify` is incorrect: Expected `bool`, found `Unknown | list[Unknown | str] | bool | str | ((href) -> Unknown)`
+ tornado/test/escape_test.py:217:51: error[invalid-argument-type] Argument to function `linkify` is incorrect: Expected `list[str]`, found `Unknown | list[Unknown | str] | bool | str | ((href) -> Unknown)`
- tornado/test/import_test.py:66:23: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `ioloop`
- tornado/test/import_test.py:66:52: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `util`
- tornado/test/import_test.py:67:23: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `gen`
- tornado/test/import_test.py:67:49: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `util`
- tornado/test/import_test.py:68:23: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `util`
- tornado/test/util_test.py:296:56: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `escape`
- tornado/test/util_test.py:302:56: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `escape`
- tornado/web.py:1380:25: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `locale`
- tornado/web.py:1401:29: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `locale`
- tornado/web.py:1404:43: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `locale`
- tornado/web.py:1414:61: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `locale`
- tornado/web.py:2242:24: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `netutil`
- tornado/websocket.py:139:24: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `web`
- tornado/websocket.py:222:22: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `web`
- tornado/websocket.py:364:23: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `escape`
- tornado/websocket.py:758:39: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `web`
- tornado/websocket.py:1097:23: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `escape`
- tornado/websocket.py:1098:19: error[unresolved-attribute] Type `<module 'tornado'>` has no attribute `escape`
- Found 244 diagnostics
+ Found 226 diagnostics

dragonchain (https://github.com/dragonchain/dragonchain)
- dragonchain/lib/dao/transaction_dao.py:77:16: error[unresolved-attribute] Type `<module 'redis'>` has no attribute `exceptions`
- dragonchain/lib/database/redis.py:364:48: error[unresolved-attribute] Type `<module 'redis'>` has no attribute `client`
- dragonchain/lib/database/redisearch.py:184:12: error[unresolved-attribute] Type `<module 'redis'>` has no attribute `exceptions`
- dragonchain/lib/database/redisearch.py:347:12: error[unresolved-attribute] Type `<module 'redis'>` has no attribute `exceptions`
- dragonchain/lib/database/redisearch.py:361:24: error[unresolved-attribute] Type `<module 'redis'>` has no attribute `exceptions`
- dragonchain/lib/database/redisearch.py:382:12: error[unresolved-attribute] Type `<module 'redis'>` has no attribute `exceptions`
- dragonchain/lib/database/redisearch.py:431:12: error[unresolved-attribute] Type `<module 'redis'>` has no attribute `exceptions`
- dragonchain/lib/database/redisearch.py:436:12: error[unresolved-attribute] Type `<module 'redis'>` has no attribute `exceptions`
- dragonchain/lib/database/redisearch.py:449:16: error[unresolved-attribute] Type `<module 'redis'>` has no attribute `exceptions`
- dragonchain/lib/database/redisearch_utest.py:84:49: error[unresolved-attribute] Type `<module 'redis'>` has no attribute `exceptions`
- dragonchain/webserver/lib/blocks.py:52:12: error[unresolved-attribute] Type `<module 'redis'>` has no attribute `exceptions`
- dragonchain/webserver/lib/transactions.py:68:12: error[unresolved-attribute] Type `<module 'redis'>` has no attribute `exceptions`
- Found 305 diagnostics
+ Found 293 diagnostics

cki-lib (https://gitlab.com/cki-project/cki-lib)
- cki_lib/timeout.py:23:20: error[unresolved-attribute] Type `<module 'multiprocessing'>` has no attribute `context`
- Found 182 diagnostics
+ Found 181 diagnostics

zulip (https://github.com/zulip/zulip)
- scripts/setup/generate_secrets.py:177:31: error[unresolved-attribute] Type `<module 'redis'>` has no attribute `exceptions`
- Found 2707 diagnostics
+ Found 2706 diagnostics

scikit-learn (https://github.com/scikit-learn/scikit-learn)
+ sklearn/manifold/_spectral_embedding.py:415:28: error[call-non-callable] Object of type `<module 'scipy.sparse.linalg._eigen.lobpcg'>` is not callable
+ sklearn/manifold/_spectral_embedding.py:449:32: error[call-non-callable] Object of type `<module 'scipy.sparse.linalg._eigen.lobpcg'>` is not callable
+ sklearn/manifold/tests/test_spectral_embedding.py:497:46: warning[possibly-missing-attribute] Attribute `__qualname__` on type `(Overload[(A: Unknown, k: int = Literal[6], M: Unknown | None = None, sigma: int | float | floating[Any] | ... omitted 3 union elements = None, which: @Todo = Literal["LM"], v0: @Todo | None = None, ncv: int | None = None, maxiter: int | None = None, tol: int | float = Literal[0], return_eigenvectors: @Todo | Literal[1] = Literal[True], Minv: Unknown | None = None, OPinv: Unknown | None = None, mode: @Todo = Literal["normal"]) -> tuple[@Todo, @Todo], (A: Unknown, k: int = Literal[6], M: Unknown | None = None, sigma: int | float | floating[Any] | ... omitted 3 union elements = None, which: @Todo = Literal["LM"], v0: @Todo | None = None, ncv: int | None = None, maxiter: int | None = None, tol: int | float = Literal[0], return_eigenvectors: @Todo | Literal[1] = Literal[True], Minv: Unknown | None = None, OPinv: Unknown | None = None, mode: @Todo = Literal["normal"]) -> tuple[@Todo, @Todo], (A: Unknown, k: int = Literal[6], M: Unknown | None = None, sigma: int | float | floating[Any] | ... omitted 3 union elements = None, which: @Todo = Literal["LM"], v0: @Todo | None = None, ncv: int | None = None, maxiter: int | None = None, tol: int | float = Literal[0], return_eigenvectors: @Todo | Literal[1] = Literal[True], Minv: Unknown | None = None, OPinv: Unknown | None = None, mode: @Todo = Literal["normal"]) -> tuple[@Todo, @Todo], (A: Unknown, k: int, M: Unknown | None, sigma: int | float | floating[Any] | ... omitted 3 union elements, which: @Todo, v0: @Todo | None, ncv: int | None, maxiter: int | None, tol: int | float, return_eigenvectors: @Todo | Literal[0], Minv: Unknown | None = None, OPinv: Unknown | None = None, mode: @Todo = Literal["normal"]) -> @Todo, (A: Unknown, k: int = Literal[6], M: Unknown | None = None, sigma: int | float | floating[Any] | ... omitted 3 union elements = None, which: @Todo = Literal["LM"], v0: @Todo | None = None, ncv: int | None = None, maxiter: int | None = None, tol: int | float = Literal[0], *, return_eigenvectors: @Todo | Literal[0], Minv: Unknown | None = None, OPinv: Unknown | None = None, mode: @Todo = Literal["normal"]) -> @Todo]) | <module 'scipy.sparse.linalg._eigen.lobpcg'>` may be missing
- Found 1993 diagnostics
+ Found 1996 diagnostics
Memory usage changes were detected when running on open source projects
prefect (https://github.com/PrefectHQ/prefect)
-     memo fields = ~366MB
+     memo fields = ~384MB

Copy link

codspeed-hq bot commented Oct 14, 2025

CodSpeed Performance Report

Merging #20855 will degrade performances by 11.02%

Comparing gankra/implort3 (1b0fe31) with main (5e08e54)

Summary

❌ 5 regressions
✅ 46 untouched

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Mode Benchmark BASE HEAD Change
WallTime medium[colour-science] 12 s 13.5 s -11.02%
WallTime medium[pandas] 33.9 s 37.8 s -10.17%
WallTime medium[static-frame] 9.5 s 10.2 s -6.92%
WallTime small[altair] 2.6 s 2.8 s -5.68%
WallTime small[freqtrade] 4.9 s 5.1 s -5.09%

Copy link
Contributor

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

@AlexWaygood AlexWaygood changed the title Support implicit imports of submodules in __init__.pyi [ty] Support implicit imports of submodules in __init__.pyi Oct 14, 2025
@AlexWaygood AlexWaygood added the ty Multi-file analysis & type inference label Oct 14, 2025
Copy link
Contributor

ecosystem-analyzer results

Lint rule Added Removed Changed
unresolved-attribute 0 61 1
invalid-argument-type 12 0 0
call-non-callable 3 0 0
possibly-missing-attribute 1 0 0
Total 16 61 1

Full report with detailed diff (timing results)

Comment on lines +277 to +279
/// The set of modules that are imported anywhere within this file.
maybe_imported_modules: Arc<FxHashSet<(u32, Option<String>, String)>>,

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this different from imported_modules?

I also suggest using a named struct here. It's non obvious what the semantic meaning of the tuple fields are.

It might be worth using Name instead of String, to reduce the amount of heap allocations (or use ModuleName?)

.iter()
// Throw out the result if the import is not a module (e.g. it's a function/class)
// So that `from a.b import myfunction` is not considered an import of `a.b`
.filter(|submodule| self.resolve_submodule(db, submodule).is_some())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we move this check into maybe_imported_relative_submodules_of_stub_package, given that maybe_imported_relative_submodules_of_stub_package is cached? Is it worth caching maybe_imported_relative_submodules_of_stub_package?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ecosystem-analyzer ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants