Skip to content

Conversation

@kylesayrs
Copy link
Collaborator

@kylesayrs kylesayrs commented Apr 22, 2025

Purpose

Changes

  • Remove meta device check, which fails if in the init_empty_weights context
  • Make execution device logic more explicit, implement fallback for modules which do not have parameters

Testing

  • Added tests to check behavior under the init_empty_weights context

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
@kylesayrs kylesayrs merged commit 8aa8b82 into main Apr 23, 2025
1 check passed
@kylesayrs kylesayrs deleted the kylesayrs/get_execution_device-meta branch April 23, 2025 18:00
Etelis added a commit to Etelis/compressed-tensors that referenced this pull request Sep 11, 2025
…a model (vllm-project#303)

* allow get_execution_device to be used when initializing a model

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>

* formatting

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>

---------

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants