Skip to content

Conversation

@kylesayrs
Copy link
Collaborator

Background

Purpose

Changes

  • Implement decompress_module_from_state_dict on DenseCompressor

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Copy link
Collaborator

@rahul-tuli rahul-tuli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we just delete the dense compressor?

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
@kylesayrs
Copy link
Collaborator Author

@rahul-tuli We can talk more about this somewhere else. I'd rather keep the/a dense compressor and get rid of the nullable property of compressors. This would make a lot of code much cleaner.

if compressor is not None:
  compressor.compress(model)

if compressor is not None:
  compressor.decompress(model)

is a lot more annoying to check every time you want to use a compressor programmatically, rather than a simply narrow type

compressor.compress(model)
compressor.decompress(model)

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
@rahul-tuli
Copy link
Collaborator

@rahul-tuli We can talk more about this somewhere else. I'd rather keep the/a dense compressor and get rid of the nullable property of compressors. This would make a lot of code much cleaner.

if compressor is not None:

  compressor.compress(model)



if compressor is not None:

  compressor.decompress(model)

is a lot more annoying to check every time you want to use a compressor programmatically, rather than a simply narrow type

compressor.compress(model)

compressor.decompress(model)

Sounds good @kylesayrs, could you create a ticket/issue so we remember to revisit this?

Approving the PR!

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
@kylesayrs kylesayrs requested a review from rahul-tuli June 9, 2025 18:50
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Copy link
Collaborator

@shanjiaz shanjiaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kylesayrs kylesayrs merged commit b3e89a2 into main Jun 9, 2025
1 check passed
@kylesayrs kylesayrs deleted the kylesayrs/hotfix-dense-compressor branch June 9, 2025 19:55
Etelis added a commit to Etelis/compressed-tensors that referenced this pull request Sep 11, 2025
vllm-project#344)

* hotfix dense compressor

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>

* fix import structure

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>

* fix import structure part 2

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>

* fix type hint

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>

* fix typehint

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>

---------

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants