Skip to content

[Transform] Spinquant with R1 and R2 #1615

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 38 commits into
base: main
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
ba617db
wip
kylesayrs Jun 6, 2025
2f5b1c8
use random-hadamard, add correctness tests
kylesayrs Jun 12, 2025
3aa35e7
add correctness test, note that precision makes a large difference
kylesayrs Jun 12, 2025
b6c088e
add on lifecycle methods
brian-dellabetta Jun 23, 2025
d1eb2a1
Merge branch 'main' into kylesayrs/transform-modifier
brian-dellabetta Jul 1, 2025
3207124
TransformModifier with SpinQuant R1&R2
brian-dellabetta Jul 2, 2025
a88ca3c
spinquant and quip_online, running but outputting gibberish
brian-dellabetta Jul 2, 2025
5bd51df
updated example
brian-dellabetta Jul 2, 2025
3c216dd
DummyModel script
brian-dellabetta Jul 8, 2025
bbcdc8c
implement fuse_norm_linears
kylesayrs Jul 10, 2025
bd7f4d5
Merge branch 'kylesayrs/fuse-helpers' into bdellabe/transform-modifier
kylesayrs Jul 10, 2025
f5c2150
R1 working
kylesayrs Jul 11, 2025
dc5c30c
add r2, increase precision
kylesayrs Jul 11, 2025
7172c26
spinquant modifier
kylesayrs Jul 11, 2025
9298e82
remove space
kylesayrs Jul 11, 2025
f77226d
use iterable
kylesayrs Jul 11, 2025
fdb64b5
add rotation validation
kylesayrs Jul 11, 2025
5daa2d5
embedding fusion
kylesayrs Jul 11, 2025
0e9af7b
add missing norm fusion
kylesayrs Jul 12, 2025
fce83be
use norm mappings
kylesayrs Jul 12, 2025
a979f8a
break into separate files
kylesayrs Jul 12, 2025
4cab29e
small cleanup
kylesayrs Jul 12, 2025
f1cc987
cleanup
kylesayrs Jul 14, 2025
a7bb2e2
more cleanup
kylesayrs Jul 14, 2025
0cf0188
make new weight on cpu
kylesayrs Jul 14, 2025
53ea307
standardize, make modifier serializable
kylesayrs Jul 14, 2025
4b4257f
add compress model script
kylesayrs Jul 14, 2025
dc7ac1a
use untie_word_embeddings
kylesayrs Jul 15, 2025
8542f8d
style
kylesayrs Jul 15, 2025
b1e637e
better registery logic
kylesayrs Jul 15, 2025
b44ac81
remove dummy model test (add later)
kylesayrs Jul 15, 2025
7a52b71
docstring
kylesayrs Jul 15, 2025
f4d7ec6
update docstring
kylesayrs Jul 15, 2025
f18d0e8
rename example file
kylesayrs Jul 15, 2025
cec2914
use match_modules_set
kylesayrs Jul 16, 2025
f6c797e
Merge branch 'main' into bdellabe/transform-modifier
brian-dellabetta Jul 16, 2025
0c5c514
unit test fixes
brian-dellabetta Jul 17, 2025
f2ef7cf
style fixes
brian-dellabetta Jul 17, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 18 additions & 14 deletions src/llmcompressor/modifiers/transform/spinquant/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,24 +55,34 @@ class SpinQuantModifier(Modifier, use_enum_values=True):
A mapping will be inferred if None is provided
:param norm_mappings: Specifies layers within a model to target for norm fusing.
A mapping will be inferred if None is provided
:param transform_config: Optional transform config which overrides `mappings`
:param transform_config: Optional transform config for overriding provided arguments
"""

rotations: List[SpinquantRotation] = Field(default_factory=lambda: ["R1", "R2"])
rotations: List[SpinquantRotation] = Field(
default_factory=lambda: ["R1", "R2"], exclude=True
)
transform_type: Literal["hadamard", "random-hadamard", "random-matrix"] = Field(
default="hadamard"
default="hadamard", exclude=True
)
randomize: bool = Field(default=False)
learnable: bool = Field(default=False)
randomize: bool = Field(default=False, exclude=True)
learnable: bool = Field(default=False, exclude=True)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kylesayrs why are we excluding these? wouldn't we want them to persist in json?


# norm mappings separate from spinquant mappings to allow users to
# override spinquant mappings with transform_config without overriding norms
mappings: Optional[SpinQuantMapping] = Field(default=None, exclude=True)
norm_mappings: Optional[List[NormMapping]] = Field(default=None, exclude=True)
mappings: Optional[SpinQuantMapping] = Field(
default=None,
repr=False,
exclude=True,
)
norm_mappings: Optional[List[NormMapping]] = Field(
default=None,
repr=False,
exclude=True,
)

# optional override for more fine-grained control
# also included in recipe serialization
transform_config: Optional[TransformConfig] = Field(default=None)
transform_config: Optional[TransformConfig] = Field(default=None, repr=False)

@field_validator("randomize", "learnable", mode="before")
def validate_not_implemented(cls, value, info: ValidationInfo):
Expand All @@ -86,12 +96,6 @@ def validate_rotations(cls, value):

def on_initialize(self, state: State, **kwargs) -> bool:
if self.transform_config is not None:
if self.mappings is not None:
raise ValueError(
"Please provide either `transform_config` or `mappings` "
"but not both"
)

return True

self.mappings = infer_mapping_from_model(state.model)
Expand Down
Loading