[Transform] Support applying transforms to embeddings #385

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

kylesayrs wants to merge 1 commit into kylesayrs/transform-attention-head from kylesayrs/transform-support-embedding

Contributor

kylesayrs commented Jul 9, 2025 •

edited

Loading

Purpose

Support applying transforms to embedding modules
- Specifically, applying R1 to the embedding module of llama models

Prerequisites

[Transform] Implement multi-headed transforms #383

Changes

Remove assertions for torch.nn.Linear module types
Add apply functions for embedding module type

Testing

Added tests for applying transforms to embedding modules followed by linear modules


          support embeddings

0b4fdb3

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet